2024-08-20T06:22:01.919 INFO:root:teuthology version: 1.2.0 2024-08-20T06:22:01.944 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-20T06:22:02.042 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546 branch: wip-hemanth-testing-2024-08-19-1357 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/blogbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7861546' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-08-19-1357 ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: mds: mds_export_ephemeral_random_max: 0.1 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: ef4275410d8483698016fe484b2a3c664080b4c1 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: ef4275410d8483698016fe484b2a3c664080b4c1 kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-08-19-1357 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 owner: scheduled_hyelloji@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7244 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 sleep_before_teardown: 0 suite: fs suite_branch: wip-hemanth-testing-2024-08-19-1357 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ef4275410d8483698016fe484b2a3c664080b4c1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOg+B6peUkmTJ+YQXEFhsrrPC9VZ/hZPn8AZ11GUzGNDOqNRm1d/1l+Yl14TorfA/y4MRgjpYJPNPT0BdL2YGO4= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrx8srCbvpUcRb9sBVqnOUTLEBh9JBeHScJRwkBLlWbMaZa5p6D3hxbiFWSHF8guVIAT441saRGRtLgudncf8E= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH78c15FciEMLfiSxlBxwlrtIgPe3cGddzcjtn+mMxfC1YYeiRJl51/WcJizN42mdcQgLvGiAlha2IgLmGFj42M= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/blogbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-19_19:57:23 tube: smithi user: hyelloji verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-20T06:22:02.043 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa; will attempt to use it 2024-08-20T06:22:02.044 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks 2024-08-20T06:22:02.044 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-20T06:22:02.045 INFO:teuthology.task.internal:Checking packages... 2024-08-20T06:22:02.066 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ef4275410d8483698016fe484b2a3c664080b4c1' 2024-08-20T06:22:02.066 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-20T06:22:02.066 INFO:teuthology.packaging:ref: None 2024-08-20T06:22:02.066 INFO:teuthology.packaging:tag: None 2024-08-20T06:22:02.067 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-08-19-1357 2024-08-20T06:22:02.067 INFO:teuthology.packaging:sha1: ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:22:02.067 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-08-19-1357 2024-08-20T06:22:02.205 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5872.gef427541 2024-08-20T06:22:02.207 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-20T06:22:02.212 INFO:teuthology.task.internal:no buildpackages task found 2024-08-20T06:22:02.212 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-20T06:22:02.230 INFO:teuthology.task.internal:Saving configuration 2024-08-20T06:22:02.248 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-20T06:22:02.254 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-20T06:22:02.279 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 06:17:02.570813', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOg+B6peUkmTJ+YQXEFhsrrPC9VZ/hZPn8AZ11GUzGNDOqNRm1d/1l+Yl14TorfA/y4MRgjpYJPNPT0BdL2YGO4='} 2024-08-20T06:22:02.303 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 06:17:02.567072', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBrx8srCbvpUcRb9sBVqnOUTLEBh9JBeHScJRwkBLlWbMaZa5p6D3hxbiFWSHF8guVIAT441saRGRtLgudncf8E='} 2024-08-20T06:22:02.327 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 06:17:02.569082', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH78c15FciEMLfiSxlBxwlrtIgPe3cGddzcjtn+mMxfC1YYeiRJl51/WcJizN42mdcQgLvGiAlha2IgLmGFj42M='} 2024-08-20T06:22:02.327 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-20T06:22:02.333 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-20T06:22:02.333 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-20T06:22:02.333 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-20T06:22:02.333 INFO:teuthology.run_tasks:Running task console_log... 2024-08-20T06:22:02.416 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7eff38d570a0>, signals=[15]) 2024-08-20T06:22:02.417 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-20T06:22:02.422 INFO:teuthology.task.internal:Opening connections... 2024-08-20T06:22:02.422 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-08-20T06:22:02.424 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:22:02.516 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-08-20T06:22:02.517 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:22:02.597 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-08-20T06:22:02.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:22:02.676 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-20T06:22:02.682 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-08-20T06:22:02.701 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-08-20T06:22:02.701 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-08-20T06:22:02.759 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-08-20T06:22:02.759 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-08-20T06:22:02.759 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-08-20T06:22:02.759 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T06:22:02.760 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T06:22:02.761 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-08-20T06:22:02.778 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-08-20T06:22:02.797 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-08-20T06:22:02.797 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-08-20T06:22:02.854 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-08-20T06:22:02.854 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-08-20T06:22:02.854 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-08-20T06:22:02.854 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T06:22:02.855 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T06:22:02.855 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-08-20T06:22:02.878 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-08-20T06:22:02.897 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-08-20T06:22:02.897 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-08-20T06:22:02.955 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-08-20T06:22:02.955 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-08-20T06:22:02.955 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-08-20T06:22:02.955 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T06:22:02.956 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T06:22:02.957 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-08-20T06:22:02.995 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-20T06:22:03.005 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-20T06:22:03.010 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-20T06:22:03.010 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-08-20T06:22:03.012 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-08-20T06:22:03.014 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-08-20T06:22:03.031 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-20T06:22:03.037 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-20T06:22:03.037 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-08-20T06:22:03.070 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-08-20T06:22:03.072 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-08-20T06:22:03.105 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-20T06:22:03.179 INFO:teuthology.run_tasks:Running task kernel... 2024-08-20T06:22:03.192 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-08-20T06:22:03.192 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-08-20T06:22:03.193 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T06:22:03.193 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T06:22:03.194 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T06:22:03.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:03.210 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-08-20T06:22:03.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:03.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:03.212 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-08-20T06:22:03.213 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-08-20T06:22:03.271 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-496.el9.x86_64 2024-08-20T06:22:03.271 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-496.el9.x86_64 2024-08-20T06:22:03.271 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2024-08-20T06:22:03.273 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-496.el9.x86_64 2024-08-20T06:22:03.273 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-496.el9.x86_64 2024-08-20T06:22:03.273 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-496.el9.x86_64 2024-08-20T06:22:03.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-08-20T06:22:03.273 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-496.el9.x86_64 2024-08-20T06:22:03.273 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-08-20T06:22:06.887 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-08-20T06:22:07.082 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-08-20T06:22:07.129 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-08-20T06:22:08.783 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-08-20T06:22:08.838 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-08-20T06:22:09.663 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-08-20T06:22:13.250 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.6 MB 00:00 2024-08-20T06:22:13.997 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.6 MB 00:00 2024-08-20T06:22:14.305 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 6.6 MB 00:01 2024-08-20T06:22:15.581 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 72 kB/s | 18 kB 00:00 2024-08-20T06:22:15.955 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 18 kB 00:00 2024-08-20T06:22:16.593 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 25 MB/s | 22 MB 00:00 2024-08-20T06:22:16.726 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 34 MB/s | 22 MB 00:00 2024-08-20T06:22:21.970 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-20T06:22:22.007 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-08-20T06:22:22.193 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:01 ago on Tue 20 Aug 2024 06:22:21 AM UTC. 2024-08-20T06:22:23.860 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-20T06:22:23.860 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-496.el9.x86_64 is already installed. 2024-08-20T06:22:23.936 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-08-20T06:22:23.937 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-08-20T06:22:23.937 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-08-20T06:22:23.973 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-20T06:22:23.973 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-496.el9.x86_64 is already installed. 2024-08-20T06:22:24.042 DEBUG:teuthology.orchestra.run.smithi107:> echo no | sudo yum reinstall kernel || true 2024-08-20T06:22:24.061 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-08-20T06:22:24.062 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-08-20T06:22:24.062 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-08-20T06:22:24.167 DEBUG:teuthology.orchestra.run.smithi183:> echo no | sudo yum reinstall kernel || true 2024-08-20T06:22:24.588 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:03 ago on Tue 20 Aug 2024 06:22:21 AM UTC. 2024-08-20T06:22:24.718 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:03 ago on Tue 20 Aug 2024 06:22:21 AM UTC. 2024-08-20T06:22:24.735 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:24.741 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:24.742 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-08-20T06:22:24.742 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:24.742 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:24.742 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 1.8 M 2024-08-20T06:22:24.742 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-08-20T06:22:24.743 INFO:teuthology.orchestra.run.smithi107.stderr:Operation aborted. 2024-08-20T06:22:24.798 INFO:teuthology.orchestra.run.smithi107.stdout:Is this ok [y/N]: 2024-08-20T06:22:24.798 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum reinstall -y kernel || true 2024-08-20T06:22:24.869 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-08-20T06:22:24.879 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:24.879 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:24.879 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:24.879 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 1.8 M 2024-08-20T06:22:24.880 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-08-20T06:22:24.881 INFO:teuthology.orchestra.run.smithi183.stderr:Operation aborted. 2024-08-20T06:22:24.941 INFO:teuthology.orchestra.run.smithi183.stdout:Is this ok [y/N]: 2024-08-20T06:22:24.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum reinstall -y kernel || true 2024-08-20T06:22:25.343 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:04 ago on Tue 20 Aug 2024 06:22:21 AM UTC. 2024-08-20T06:22:25.465 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalling: 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 1.8 M 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 0 2024-08-20T06:22:25.466 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-08-20T06:22:25.490 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:04 ago on Tue 20 Aug 2024 06:22:21 AM UTC. 2024-08-20T06:22:25.614 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-08-20T06:22:25.614 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalling: 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 1.8 M 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 0 2024-08-20T06:22:25.615 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-08-20T06:22:25.932 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 1.6 kB/s | 18 kB 00:11 2024-08-20T06:22:25.998 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:22:26.144 INFO:teuthology.orchestra.run.smithi183.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:22:26.288 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:22:26.434 INFO:teuthology.orchestra.run.smithi183.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:22:26.497 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-496.el9.x86_64.rpm 1.9 MB/s | 1.8 MB 00:00 2024-08-20T06:22:26.497 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:22:26.497 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.7 MB/s | 1.8 MB 00:01 2024-08-20T06:22:26.526 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-08-20T06:22:26.529 INFO:teuthology.orchestra.run.smithi183.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for http://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 141.219.188.21) 2024-08-20T06:22:26.676 INFO:teuthology.orchestra.run.smithi183.stdout:[MIRROR] kernel-5.14.0-496.el9.x86_64.rpm: Status code: 404 for https://mirrors.lug.mtu.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-496.el9.x86_64.rpm (IP: 141.219.188.21) 2024-08-20T06:22:26.790 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-08-20T06:22:26.790 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-08-20T06:22:26.818 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-08-20T06:22:26.819 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-08-20T06:22:26.892 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-496.el9.x86_64.rpm 1.5 MB/s | 1.8 MB 00:01 2024-08-20T06:22:26.893 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:22:26.894 INFO:teuthology.orchestra.run.smithi183.stdout:Total 1.4 MB/s | 1.8 MB 00:01 2024-08-20T06:22:26.925 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-08-20T06:22:26.928 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-20T06:22:26.972 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-08-20T06:22:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: Reinstalling : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:27.066 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:27.197 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-08-20T06:22:27.197 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-08-20T06:22:27.226 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-08-20T06:22:27.226 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-08-20T06:22:27.380 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-08-20T06:22:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: Reinstalling : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:27.478 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:27.785 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:28.158 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout:Reinstalled: 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:22:28.562 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-08-20T06:22:28.743 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-08-20T06:22:28.801 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:28.801 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-496.el9.x86_64 2024-08-20T06:22:28.801 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-20T06:22:28.801 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-08-20T06:22:28.801 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-20T06:22:28.830 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-08-20T06:22:28.858 DEBUG:teuthology.parallel:result is None 2024-08-20T06:22:28.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:28.916 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:28.916 INFO:teuthology.orchestra.run.smithi183.stdout:Reinstalled: 2024-08-20T06:22:28.917 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:28.917 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:22:28.917 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-08-20T06:22:29.124 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-08-20T06:22:29.181 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:29.185 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-496.el9.x86_64 2024-08-20T06:22:29.185 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-20T06:22:29.185 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-08-20T06:22:29.185 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-20T06:22:29.218 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-08-20T06:22:29.237 DEBUG:teuthology.parallel:result is None 2024-08-20T06:22:32.309 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-08-20T06:22:34.247 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-08-20T06:22:34.247 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-496.el9.x86_64 is already installed. 2024-08-20T06:22:34.328 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-08-20T06:22:34.329 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-08-20T06:22:34.329 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-08-20T06:22:34.458 DEBUG:teuthology.orchestra.run.smithi193:> echo no | sudo yum reinstall kernel || true 2024-08-20T06:22:35.009 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:03 ago on Tue 20 Aug 2024 06:22:32 AM UTC. 2024-08-20T06:22:35.161 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.173 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:35.174 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 1.8 M 2024-08-20T06:22:35.174 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-08-20T06:22:35.174 INFO:teuthology.orchestra.run.smithi193.stderr:Operation aborted. 2024-08-20T06:22:35.245 INFO:teuthology.orchestra.run.smithi193.stdout:Is this ok [y/N]: 2024-08-20T06:22:35.245 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum reinstall -y kernel || true 2024-08-20T06:22:35.793 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:03 ago on Tue 20 Aug 2024 06:22:32 AM UTC. 2024-08-20T06:22:35.915 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-496.el9 baseos 1.8 M 2024-08-20T06:22:35.916 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 1.8 M 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-08-20T06:22:35.917 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-08-20T06:22:36.235 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-496.el9.x86_64.rpm 8.7 MB/s | 1.8 MB 00:00 2024-08-20T06:22:36.235 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:22:36.235 INFO:teuthology.orchestra.run.smithi193.stdout:Total 5.6 MB/s | 1.8 MB 00:00 2024-08-20T06:22:36.263 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-08-20T06:22:36.518 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-08-20T06:22:36.518 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-08-20T06:22:36.547 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-08-20T06:22:36.548 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-08-20T06:22:36.726 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-08-20T06:22:36.819 INFO:teuthology.orchestra.run.smithi193.stdout: Reinstalling : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:36.820 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:37.596 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:37.596 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 1/2 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-496.el9.x86_64 2/2 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalled: 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:22:38.389 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-08-20T06:22:38.573 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-08-20T06:22:38.630 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-496.el9.x86_64 2024-08-20T06:22:38.631 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-496.el9.x86_64 2024-08-20T06:22:38.631 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-08-20T06:22:38.631 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-08-20T06:22:38.631 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-08-20T06:22:38.673 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-08-20T06:22:38.696 DEBUG:teuthology.parallel:result is None 2024-08-20T06:22:38.696 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-20T06:22:38.703 INFO:teuthology.task.internal:Creating test directory... 2024-08-20T06:22:38.703 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T06:22:38.706 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T06:22:38.709 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T06:22:38.751 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-20T06:22:38.758 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-20T06:22:38.765 INFO:teuthology.task.internal:Creating archive directory... 2024-08-20T06:22:38.765 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T06:22:38.767 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T06:22:38.769 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T06:22:38.871 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-20T06:22:38.880 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-20T06:22:38.880 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T06:22:38.882 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T06:22:38.884 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T06:22:38.911 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.914 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.923 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.924 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.937 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.948 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T06:22:38.950 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-20T06:22:38.957 INFO:teuthology.task.internal:Configuring sudo... 2024-08-20T06:22:38.957 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T06:22:38.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T06:22:38.970 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T06:22:38.996 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-20T06:22:39.006 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-20T06:22:39.006 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T06:22:39.036 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T06:22:39.038 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T06:22:39.054 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.181 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.306 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:22:39.306 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T06:22:39.371 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.416 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.467 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.550 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.596 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:22:39.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T06:22:39.664 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.706 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T06:22:39.756 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.840 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T06:22:39.879 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:22:39.880 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T06:22:39.944 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-08-20T06:22:39.947 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-08-20T06:22:39.949 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-08-20T06:22:40.001 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T06:22:40.005 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T06:22:40.028 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T06:22:40.397 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-20T06:22:40.406 INFO:teuthology.task.internal:Starting timer... 2024-08-20T06:22:40.406 INFO:teuthology.run_tasks:Running task pcp... 2024-08-20T06:22:40.416 INFO:teuthology.run_tasks:Running task selinux... 2024-08-20T06:22:40.424 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-20T06:22:40.424 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-08-20T06:22:40.482 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-08-20T06:22:40.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-08-20T06:22:40.535 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-08-20T06:22:40.537 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-08-20T06:22:40.578 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-08-20T06:22:40.580 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-20T06:22:40.580 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-08-20T06:22:40.605 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-08-20T06:22:40.606 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-08-20T06:22:40.627 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-08-20T06:22:40.627 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-08-20T06:22:40.650 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-08-20T06:22:40.650 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-08-20T06:22:40.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T06:22:40.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:40.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T06:22:40.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:40.704 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T06:22:40.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:22:40.730 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-20T06:22:40.730 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-08-20T06:22:40.771 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-08-20T06:22:40.798 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-08-20T06:22:40.833 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-20T06:22:40.843 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-20T06:22:40.855 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-20T06:22:40.856 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi183.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-20T06:24:23.984 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-08-20T06:24:23.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-08-20T06:24:23.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:24:24.062 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-08-20T06:24:24.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-08-20T06:24:24.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-08-20T06:24:24.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:24:24.222 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-08-20T06:24:24.309 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-08-20T06:24:24.310 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-08-20T06:24:24.310 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T06:24:24.386 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-08-20T06:24:24.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-08-20T06:24:24.471 INFO:teuthology.run_tasks:Running task clock... 2024-08-20T06:24:24.482 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-20T06:24:24.482 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T06:24:24.483 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T06:24:24.485 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T06:24:24.485 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T06:24:24.487 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T06:24:24.487 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T06:24:24.511 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T06:24:24.518 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T06:24:24.525 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T06:24:24.534 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T06:24:24.542 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T06:24:24.549 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-08-20T06:24:24.558 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T06:24:24.560 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-08-20T06:24:24.561 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-08-20T06:24:24.573 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-08-20T06:24:24.575 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T06:24:24.583 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-08-20T06:24:24.588 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T06:24:24.589 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T06:24:24.595 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-08-20T06:24:24.603 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T06:24:24.610 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T06:24:24.624 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T06:24:24.641 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-08-20T06:24:24.644 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T06:24:24.644 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-08-20T06:24:24.644 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.645 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.645 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.645 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.667 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-08-20T06:24:24.670 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T06:24:24.670 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-08-20T06:24:24.670 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.671 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.671 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.671 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.679 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-08-20T06:24:24.683 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T06:24:24.683 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-08-20T06:24:24.683 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.683 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.684 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.684 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T06:24:24.684 INFO:teuthology.run_tasks:Running task install... 2024-08-20T06:24:24.692 DEBUG:teuthology.task.install:project ceph 2024-08-20T06:24:24.692 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'}} 2024-08-20T06:24:24.692 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'} 2024-08-20T06:24:24.692 INFO:teuthology.task.install:Using flavor: default 2024-08-20T06:24:24.698 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-20T06:24:24.698 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-20T06:24:24.698 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T06:24:24.699 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:24:24.701 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T06:24:24.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:24:24.702 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T06:24:24.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:24:24.848 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T06:24:24.848 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T06:24:24.856 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T06:24:24.856 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T06:24:24.872 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T06:24:24.872 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T06:24:24.961 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T06:24:24.961 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:24:24.961 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T06:24:24.964 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T06:24:24.964 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:24:24.964 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T06:24:24.979 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T06:24:24.979 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:24:24.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T06:24:24.990 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T06:24:24.990 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T06:24:24.991 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T06:24:24.991 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T06:24:25.006 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T06:24:25.006 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T06:24:25.056 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T06:24:25.058 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T06:24:25.071 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T06:24:25.131 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T06:24:25.133 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T06:24:25.146 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T06:24:25.195 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-08-20T06:24:25.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-08-20T06:24:25.198 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-08-20T06:24:25.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-08-20T06:24:25.210 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-08-20T06:24:25.212 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-08-20T06:24:25.420 INFO:teuthology.orchestra.run.smithi193.stdout:45 files removed 2024-08-20T06:24:25.426 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-08-20T06:24:25.436 INFO:teuthology.orchestra.run.smithi183.stdout:45 files removed 2024-08-20T06:24:25.446 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T06:24:25.452 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T06:24:25.463 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T06:24:25.985 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 318 kB/s | 84 kB 00:00 2024-08-20T06:24:25.993 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-08-20T06:24:26.013 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-08-20T06:24:26.276 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-08-20T06:24:26.277 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-08-20T06:24:26.313 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 94 kB/s | 24 kB 00:00 2024-08-20T06:24:26.544 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-08-20T06:24:26.561 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-08-20T06:24:26.616 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 7.3 kB/s | 1.9 kB 00:00 2024-08-20T06:24:27.049 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.2 MB 00:00 2024-08-20T06:24:27.219 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-08-20T06:24:28.378 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-08-20T06:24:29.187 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-08-20T06:24:30.117 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-08-20T06:24:34.310 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.6 MB 00:00 2024-08-20T06:24:34.696 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 3.9 MB/s | 6.6 MB 00:01 2024-08-20T06:24:36.130 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 107 kB/s | 18 kB 00:00 2024-08-20T06:24:36.235 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 18 kB 00:00 2024-08-20T06:24:37.100 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-20T06:24:37.114 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 29 MB/s | 22 MB 00:00 2024-08-20T06:24:42.453 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-08-20T06:24:42.523 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-08-20T06:24:44.707 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.708 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.719 INFO:teuthology.orchestra.run.smithi107.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T06:24:44.720 INFO:teuthology.orchestra.run.smithi107.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.721 INFO:teuthology.orchestra.run.smithi107.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T06:24:44.721 INFO:teuthology.orchestra.run.smithi107.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.722 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.723 INFO:teuthology.orchestra.run.smithi107.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T06:24:44.723 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.725 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.726 INFO:teuthology.orchestra.run.smithi107.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.735 INFO:teuthology.orchestra.run.smithi183.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T06:24:44.735 INFO:teuthology.orchestra.run.smithi183.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T06:24:44.736 INFO:teuthology.orchestra.run.smithi183.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T06:24:44.737 INFO:teuthology.orchestra.run.smithi183.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.739 INFO:teuthology.orchestra.run.smithi183.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T06:24:44.741 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.742 INFO:teuthology.orchestra.run.smithi183.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T06:24:44.787 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-08-20T06:24:44.795 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-08-20T06:24:44.795 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T06:24:44.796 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T06:24:44.797 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T06:24:44.798 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T06:24:44.799 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T06:24:44.800 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:24:44.801 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T06:24:44.802 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T06:24:44.803 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T06:24:44.804 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T06:24:44.805 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T06:24:44.806 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T06:24:44.807 INFO:teuthology.orchestra.run.smithi107.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:Install 148 Packages 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-08-20T06:24:44.808 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:24:44.809 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-08-20T06:24:44.809 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 232 M 2024-08-20T06:24:44.810 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-08-20T06:24:44.810 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-08-20T06:24:44.810 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T06:24:44.811 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T06:24:44.812 INFO:teuthology.orchestra.run.smithi183.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T06:24:44.813 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T06:24:44.814 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T06:24:44.815 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T06:24:44.816 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T06:24:44.817 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T06:24:44.818 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T06:24:44.819 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T06:24:44.820 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T06:24:44.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-08-20T06:24:44.822 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:Install 148 Packages 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 2 Packages 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 232 M 2024-08-20T06:24:44.823 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-08-20T06:24:45.171 INFO:teuthology.orchestra.run.smithi183.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-08-20T06:24:45.363 INFO:teuthology.orchestra.run.smithi183.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 4.6 MB/s | 901 kB 00:00 2024-08-20T06:24:45.422 INFO:teuthology.orchestra.run.smithi183.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 2.4 MB/s | 145 kB 00:00 2024-08-20T06:24:45.506 INFO:teuthology.orchestra.run.smithi183.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 12 MB/s | 5.5 MB 00:00 2024-08-20T06:24:45.591 INFO:teuthology.orchestra.run.smithi107.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 42 kB/s | 6.6 kB 00:00 2024-08-20T06:24:45.622 INFO:teuthology.orchestra.run.smithi183.stdout:(5/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 13 MB/s | 1.5 MB 00:00 2024-08-20T06:24:45.664 INFO:teuthology.orchestra.run.smithi183.stdout:(6/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 9.8 MB/s | 2.4 MB 00:00 2024-08-20T06:24:45.940 INFO:teuthology.orchestra.run.smithi183.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 15 MB/s | 4.8 MB 00:00 2024-08-20T06:24:46.116 INFO:teuthology.orchestra.run.smithi107.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 901 kB 00:00 2024-08-20T06:24:46.275 INFO:teuthology.orchestra.run.smithi107.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 917 kB/s | 145 kB 00:00 2024-08-20T06:24:46.406 INFO:teuthology.orchestra.run.smithi183.stdout:(8/150): ceph-common-19.0.0-5872.gef427541.el9. 16 MB/s | 22 MB 00:01 2024-08-20T06:24:46.440 INFO:teuthology.orchestra.run.smithi183.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 762 kB/s | 25 kB 00:00 2024-08-20T06:24:46.699 INFO:teuthology.orchestra.run.smithi183.stdout:(10/150): ceph-radosgw-19.0.0-5872.gef427541.el 14 MB/s | 11 MB 00:00 2024-08-20T06:24:46.775 INFO:teuthology.orchestra.run.smithi107.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 4.1 MB/s | 5.5 MB 00:01 2024-08-20T06:24:46.858 INFO:teuthology.orchestra.run.smithi107.stdout:(5/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-08-20T06:24:46.890 INFO:teuthology.orchestra.run.smithi183.stdout:(11/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 14 MB/s | 17 MB 00:01 2024-08-20T06:24:46.907 INFO:teuthology.orchestra.run.smithi183.stdout:(12/150): cephfs-mirror-19.0.0-5872.gef427541.e 1.1 MB/s | 225 kB 00:00 2024-08-20T06:24:46.949 INFO:teuthology.orchestra.run.smithi183.stdout:(13/150): cephfs-shell-19.0.0-5872.gef427541.el 464 kB/s | 27 kB 00:00 2024-08-20T06:24:46.950 INFO:teuthology.orchestra.run.smithi107.stdout:(6/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-08-20T06:24:46.974 INFO:teuthology.orchestra.run.smithi183.stdout:(14/150): libcephfs-devel-19.0.0-5872.gef427541 474 kB/s | 32 kB 00:00 2024-08-20T06:24:47.041 INFO:teuthology.orchestra.run.smithi183.stdout:(15/150): libcephfs2-19.0.0-5872.gef427541.el9. 8.0 MB/s | 755 kB 00:00 2024-08-20T06:24:47.066 INFO:teuthology.orchestra.run.smithi183.stdout:(16/150): libcephsqlite-19.0.0-5872.gef427541.e 1.7 MB/s | 163 kB 00:00 2024-08-20T06:24:47.108 INFO:teuthology.orchestra.run.smithi183.stdout:(17/150): librados-devel-19.0.0-5872.gef427541. 1.8 MB/s | 126 kB 00:00 2024-08-20T06:24:47.150 INFO:teuthology.orchestra.run.smithi183.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 5.9 MB/s | 508 kB 00:00 2024-08-20T06:24:47.242 INFO:teuthology.orchestra.run.smithi183.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 492 kB/s | 45 kB 00:00 2024-08-20T06:24:47.343 INFO:teuthology.orchestra.run.smithi183.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 1.4 MB/s | 138 kB 00:00 2024-08-20T06:24:47.426 INFO:teuthology.orchestra.run.smithi183.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 2.2 MB/s | 188 kB 00:00 2024-08-20T06:24:47.527 INFO:teuthology.orchestra.run.smithi183.stdout:(22/150): librgw2-19.0.0-5872.gef427541.el9.x86 13 MB/s | 5.5 MB 00:00 2024-08-20T06:24:47.552 INFO:teuthology.orchestra.run.smithi183.stdout:(23/150): python3-rados-19.0.0-5872.gef427541.e 2.9 MB/s | 371 kB 00:00 2024-08-20T06:24:47.610 INFO:teuthology.orchestra.run.smithi183.stdout:(24/150): python3-rbd-19.0.0-5872.gef427541.el9 4.2 MB/s | 355 kB 00:00 2024-08-20T06:24:47.617 INFO:teuthology.orchestra.run.smithi107.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 6.3 MB/s | 4.8 MB 00:00 2024-08-20T06:24:47.636 INFO:teuthology.orchestra.run.smithi183.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 1.4 MB/s | 116 kB 00:00 2024-08-20T06:24:47.686 INFO:teuthology.orchestra.run.smithi183.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.1 MB/s | 85 kB 00:00 2024-08-20T06:24:47.928 INFO:teuthology.orchestra.run.smithi183.stdout:(27/150): rbd-mirror-19.0.0-5872.gef427541.el9. 11 MB/s | 3.2 MB 00:00 2024-08-20T06:24:47.995 INFO:teuthology.orchestra.run.smithi183.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 515 kB/s | 34 kB 00:00 2024-08-20T06:24:48.062 INFO:teuthology.orchestra.run.smithi183.stdout:(29/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 2.2 MB/s | 150 kB 00:00 2024-08-20T06:24:48.134 INFO:teuthology.orchestra.run.smithi107.stdout:(8/150): ceph-common-19.0.0-5872.gef427541.el9. 8.0 MB/s | 22 MB 00:02 2024-08-20T06:24:48.160 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 20 MB 00:18 2024-08-20T06:24:48.209 INFO:teuthology.orchestra.run.smithi107.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 336 kB/s | 25 kB 00:00 2024-08-20T06:24:48.421 INFO:teuthology.orchestra.run.smithi183.stdout:(30/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 231 kB/s | 170 kB 00:00 2024-08-20T06:24:48.426 INFO:teuthology.orchestra.run.smithi107.stdout:(10/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 12 MB/s | 17 MB 00:01 2024-08-20T06:24:48.552 INFO:teuthology.orchestra.run.smithi107.stdout:(11/150): cephfs-mirror-19.0.0-5872.gef427541.e 1.8 MB/s | 225 kB 00:00 2024-08-20T06:24:48.652 INFO:teuthology.orchestra.run.smithi107.stdout:(12/150): cephfs-shell-19.0.0-5872.gef427541.el 270 kB/s | 27 kB 00:00 2024-08-20T06:24:48.752 INFO:teuthology.orchestra.run.smithi107.stdout:(13/150): libcephfs-devel-19.0.0-5872.gef427541 317 kB/s | 32 kB 00:00 2024-08-20T06:24:48.903 INFO:teuthology.orchestra.run.smithi107.stdout:(14/150): ceph-radosgw-19.0.0-5872.gef427541.el 8.5 MB/s | 11 MB 00:01 2024-08-20T06:24:48.944 INFO:teuthology.orchestra.run.smithi107.stdout:(15/150): libcephfs2-19.0.0-5872.gef427541.el9. 3.8 MB/s | 755 kB 00:00 2024-08-20T06:24:48.978 INFO:teuthology.orchestra.run.smithi107.stdout:(16/150): libcephsqlite-19.0.0-5872.gef427541.e 2.1 MB/s | 163 kB 00:00 2024-08-20T06:24:48.979 INFO:teuthology.orchestra.run.smithi183.stdout:(31/150): ceph-mgr-dashboard-19.0.0-5872.gef427 15 MB/s | 14 MB 00:00 2024-08-20T06:24:49.011 INFO:teuthology.orchestra.run.smithi107.stdout:(17/150): librados-devel-19.0.0-5872.gef427541. 1.9 MB/s | 126 kB 00:00 2024-08-20T06:24:49.053 INFO:teuthology.orchestra.run.smithi107.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 6.6 MB/s | 508 kB 00:00 2024-08-20T06:24:49.054 INFO:teuthology.orchestra.run.smithi183.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 3.7 MB/s | 284 kB 00:00 2024-08-20T06:24:49.137 INFO:teuthology.orchestra.run.smithi107.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 541 kB/s | 45 kB 00:00 2024-08-20T06:24:49.205 INFO:teuthology.orchestra.run.smithi183.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 328 kB/s | 49 kB 00:00 2024-08-20T06:24:49.221 INFO:teuthology.orchestra.run.smithi107.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 1.6 MB/s | 138 kB 00:00 2024-08-20T06:24:49.297 INFO:teuthology.orchestra.run.smithi183.stdout:(34/150): ceph-mgr-diskprediction-local-19.0.0- 8.4 MB/s | 7.4 MB 00:00 2024-08-20T06:24:49.304 INFO:teuthology.orchestra.run.smithi107.stdout:(21/150): librgw2-19.0.0-5872.gef427541.el9.x86 19 MB/s | 5.5 MB 00:00 2024-08-20T06:24:49.314 INFO:teuthology.orchestra.run.smithi183.stdout:(35/150): ceph-prometheus-alerts-19.0.0-5872.ge 151 kB/s | 16 kB 00:00 2024-08-20T06:24:49.330 INFO:teuthology.orchestra.run.smithi107.stdout:(22/150): python3-cephfs-19.0.0-5872.gef427541. 1.7 MB/s | 188 kB 00:00 2024-08-20T06:24:49.347 INFO:teuthology.orchestra.run.smithi183.stdout:(36/150): ceph-volume-19.0.0-5872.gef427541.el9 5.6 MB/s | 285 kB 00:00 2024-08-20T06:24:49.355 INFO:teuthology.orchestra.run.smithi107.stdout:(23/150): python3-rados-19.0.0-5872.gef427541.e 7.2 MB/s | 371 kB 00:00 2024-08-20T06:24:49.381 INFO:teuthology.orchestra.run.smithi183.stdout:(37/150): cephadm-19.0.0-5872.gef427541.el9.noa 11 MB/s | 772 kB 00:00 2024-08-20T06:24:49.397 INFO:teuthology.orchestra.run.smithi107.stdout:(24/150): python3-rbd-19.0.0-5872.gef427541.el9 5.2 MB/s | 355 kB 00:00 2024-08-20T06:24:49.406 INFO:teuthology.orchestra.run.smithi183.stdout:(38/150): cephfs-top-19.0.0-5872.gef427541.el9. 311 kB/s | 18 kB 00:00 2024-08-20T06:24:49.422 INFO:teuthology.orchestra.run.smithi107.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 1.7 MB/s | 116 kB 00:00 2024-08-20T06:24:49.448 INFO:teuthology.orchestra.run.smithi107.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-20T06:24:49.507 INFO:teuthology.orchestra.run.smithi183.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 637 kB/s | 80 kB 00:00 2024-08-20T06:24:49.531 INFO:teuthology.orchestra.run.smithi107.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 2.0 MB/s | 170 kB 00:00 2024-08-20T06:24:49.573 INFO:teuthology.orchestra.run.smithi183.stdout:(40/150): libgfortran-11.5.0-2.el9.x86_64.rpm 4.7 MB/s | 801 kB 00:00 2024-08-20T06:24:49.599 INFO:teuthology.orchestra.run.smithi107.stdout:(28/150): rbd-mirror-19.0.0-5872.gef427541.el9. 18 MB/s | 3.2 MB 00:00 2024-08-20T06:24:49.599 INFO:teuthology.orchestra.run.smithi183.stdout:(41/150): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-08-20T06:24:49.615 INFO:teuthology.orchestra.run.smithi107.stdout:(29/150): ceph-grafana-dashboards-19.0.0-5872.g 410 kB/s | 34 kB 00:00 2024-08-20T06:24:49.629 INFO:teuthology.orchestra.run.smithi183.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.5 MB/s | 253 kB 00:00 2024-08-20T06:24:49.649 INFO:teuthology.orchestra.run.smithi107.stdout:(30/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 2.9 MB/s | 150 kB 00:00 2024-08-20T06:24:49.671 INFO:teuthology.orchestra.run.smithi183.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 17 MB/s | 1.2 MB 00:00 2024-08-20T06:24:49.688 INFO:teuthology.orchestra.run.smithi183.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-20T06:24:49.713 INFO:teuthology.orchestra.run.smithi183.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 3.1 MB/s | 135 kB 00:00 2024-08-20T06:24:49.739 INFO:teuthology.orchestra.run.smithi183.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-20T06:24:49.764 INFO:teuthology.orchestra.run.smithi183.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-08-20T06:24:49.797 INFO:teuthology.orchestra.run.smithi183.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-08-20T06:24:49.839 INFO:teuthology.orchestra.run.smithi183.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-08-20T06:24:49.865 INFO:teuthology.orchestra.run.smithi183.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-08-20T06:24:49.890 INFO:teuthology.orchestra.run.smithi183.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-08-20T06:24:49.915 INFO:teuthology.orchestra.run.smithi183.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-08-20T06:24:49.941 INFO:teuthology.orchestra.run.smithi183.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-08-20T06:24:49.982 INFO:teuthology.orchestra.run.smithi183.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-08-20T06:24:50.008 INFO:teuthology.orchestra.run.smithi107.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 21 MB/s | 7.4 MB 00:00 2024-08-20T06:24:50.033 INFO:teuthology.orchestra.run.smithi183.stdout:(55/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-08-20T06:24:50.058 INFO:teuthology.orchestra.run.smithi183.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-08-20T06:24:50.066 INFO:teuthology.orchestra.run.smithi107.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 4.8 MB/s | 284 kB 00:00 2024-08-20T06:24:50.083 INFO:teuthology.orchestra.run.smithi183.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-08-20T06:24:50.108 INFO:teuthology.orchestra.run.smithi183.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-08-20T06:24:50.125 INFO:teuthology.orchestra.run.smithi107.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 847 kB/s | 49 kB 00:00 2024-08-20T06:24:50.142 INFO:teuthology.orchestra.run.smithi183.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-08-20T06:24:50.167 INFO:teuthology.orchestra.run.smithi107.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 394 kB/s | 16 kB 00:00 2024-08-20T06:24:50.176 INFO:teuthology.orchestra.run.smithi183.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-08-20T06:24:50.201 INFO:teuthology.orchestra.run.smithi183.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-08-20T06:24:50.226 INFO:teuthology.orchestra.run.smithi183.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-08-20T06:24:50.242 INFO:teuthology.orchestra.run.smithi107.stdout:(35/150): ceph-volume-19.0.0-5872.gef427541.el9 3.7 MB/s | 285 kB 00:00 2024-08-20T06:24:50.251 INFO:teuthology.orchestra.run.smithi183.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-08-20T06:24:50.277 INFO:teuthology.orchestra.run.smithi183.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-08-20T06:24:50.302 INFO:teuthology.orchestra.run.smithi183.stdout:(65/150): openblas-0.3.26-1.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-08-20T06:24:50.328 INFO:teuthology.orchestra.run.smithi107.stdout:(36/150): cephadm-19.0.0-5872.gef427541.el9.noa 8.8 MB/s | 772 kB 00:00 2024-08-20T06:24:50.395 INFO:teuthology.orchestra.run.smithi107.stdout:(37/150): cephfs-top-19.0.0-5872.gef427541.el9. 273 kB/s | 18 kB 00:00 2024-08-20T06:24:50.444 INFO:teuthology.orchestra.run.smithi183.stdout:(66/150): openblas-openmp-0.3.26-1.el9.x86_64.r 30 MB/s | 4.9 MB 00:00 2024-08-20T06:24:50.771 INFO:teuthology.orchestra.run.smithi107.stdout:(38/150): ceph-mgr-dashboard-19.0.0-5872.gef427 12 MB/s | 14 MB 00:01 2024-08-20T06:24:50.886 INFO:teuthology.orchestra.run.smithi183.stdout:(67/150): ceph-test-19.0.0-5872.gef427541.el9.x 11 MB/s | 50 MB 00:04 2024-08-20T06:24:50.936 INFO:teuthology.orchestra.run.smithi183.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-08-20T06:24:50.995 INFO:teuthology.orchestra.run.smithi183.stdout:(69/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-08-20T06:24:51.020 INFO:teuthology.orchestra.run.smithi183.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 1.6 MB/s | 137 kB 00:00 2024-08-20T06:24:51.070 INFO:teuthology.orchestra.run.smithi183.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-08-20T06:24:51.112 INFO:teuthology.orchestra.run.smithi183.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-08-20T06:24:51.146 INFO:teuthology.orchestra.run.smithi183.stdout:(73/150): postgresql-contrib-13.16-1.el9.x86_64 3.4 MB/s | 910 kB 00:00 2024-08-20T06:24:51.230 INFO:teuthology.orchestra.run.smithi183.stdout:(74/150): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-08-20T06:24:51.246 INFO:teuthology.orchestra.run.smithi183.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 610 kB/s | 61 kB 00:00 2024-08-20T06:24:51.272 INFO:teuthology.orchestra.run.smithi183.stdout:(76/150): python3-devel-3.9.19-7.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-08-20T06:24:51.297 INFO:teuthology.orchestra.run.smithi183.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-20T06:24:51.314 INFO:teuthology.orchestra.run.smithi183.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 614 kB/s | 26 kB 00:00 2024-08-20T06:24:51.331 INFO:teuthology.orchestra.run.smithi183.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 572 kB/s | 19 kB 00:00 2024-08-20T06:24:51.356 INFO:teuthology.orchestra.run.smithi183.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-08-20T06:24:51.371 INFO:teuthology.orchestra.run.smithi107.stdout:(39/150): ceph-test-19.0.0-5872.gef427541.el9.x 16 MB/s | 50 MB 00:03 2024-08-20T06:24:51.373 INFO:teuthology.orchestra.run.smithi183.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 834 kB/s | 35 kB 00:00 2024-08-20T06:24:51.405 INFO:teuthology.orchestra.run.smithi107.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 79 kB/s | 80 kB 00:01 2024-08-20T06:24:51.415 INFO:teuthology.orchestra.run.smithi183.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-08-20T06:24:51.490 INFO:teuthology.orchestra.run.smithi183.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-08-20T06:24:51.519 INFO:teuthology.orchestra.run.smithi183.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-08-20T06:24:51.532 INFO:teuthology.orchestra.run.smithi183.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-08-20T06:24:51.549 INFO:teuthology.orchestra.run.smithi183.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-08-20T06:24:51.582 INFO:teuthology.orchestra.run.smithi183.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-20T06:24:51.600 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:24:51.649 INFO:teuthology.orchestra.run.smithi183.stdout:(88/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-08-20T06:24:51.675 INFO:teuthology.orchestra.run.smithi183.stdout:(89/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-08-20T06:24:51.700 INFO:teuthology.orchestra.run.smithi183.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-20T06:24:51.725 INFO:teuthology.orchestra.run.smithi183.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-20T06:24:51.751 INFO:teuthology.orchestra.run.smithi183.stdout:(92/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.6 MB/s | 279 kB 00:00 2024-08-20T06:24:51.776 INFO:teuthology.orchestra.run.smithi183.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-08-20T06:24:51.801 INFO:teuthology.orchestra.run.smithi183.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 886 kB/s | 44 kB 00:00 2024-08-20T06:24:51.812 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:24:51.826 INFO:teuthology.orchestra.run.smithi183.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-20T06:24:51.852 INFO:teuthology.orchestra.run.smithi183.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-08-20T06:24:51.877 INFO:teuthology.orchestra.run.smithi183.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-08-20T06:24:51.892 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:24:51.902 INFO:teuthology.orchestra.run.smithi183.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-20T06:24:51.927 INFO:teuthology.orchestra.run.smithi183.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-20T06:24:51.953 INFO:teuthology.orchestra.run.smithi183.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-20T06:24:51.978 INFO:teuthology.orchestra.run.smithi183.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-20T06:24:52.003 INFO:teuthology.orchestra.run.smithi183.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-08-20T06:24:52.063 INFO:teuthology.orchestra.run.smithi107.stdout:(41/150): libgfortran-11.5.0-2.el9.x86_64.rpm 620 kB/s | 801 kB 00:01 2024-08-20T06:24:52.105 INFO:teuthology.orchestra.run.smithi107.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:24:52.147 INFO:teuthology.orchestra.run.smithi107.stdout:(42/150): libquadmath-11.5.0-2.el9.x86_64.rpm 249 kB/s | 191 kB 00:00 2024-08-20T06:24:52.180 INFO:teuthology.orchestra.run.smithi107.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 327 kB/s | 253 kB 00:00 2024-08-20T06:24:52.425 INFO:teuthology.orchestra.run.smithi183.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-08-20T06:24:52.427 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.6 MB 00:00 2024-08-20T06:24:52.479 INFO:teuthology.orchestra.run.smithi183.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 10 MB/s | 548 kB 00:00 2024-08-20T06:24:52.504 INFO:teuthology.orchestra.run.smithi183.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-20T06:24:52.529 INFO:teuthology.orchestra.run.smithi183.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 791 kB/s | 19 kB 00:00 2024-08-20T06:24:52.638 INFO:teuthology.orchestra.run.smithi183.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-20T06:24:52.663 INFO:teuthology.orchestra.run.smithi183.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-20T06:24:52.672 INFO:teuthology.orchestra.run.smithi107.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 206 kB/s | 106 kB 00:00 2024-08-20T06:24:52.688 INFO:teuthology.orchestra.run.smithi183.stdout:(109/150): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-08-20T06:24:52.713 INFO:teuthology.orchestra.run.smithi183.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 989 kB/s | 49 kB 00:00 2024-08-20T06:24:52.739 INFO:teuthology.orchestra.run.smithi183.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 8.4 MB/s | 151 kB 00:00 2024-08-20T06:24:52.764 INFO:teuthology.orchestra.run.smithi183.stdout:(112/150): libunwind-1.6.2-1.el9.x86_64.rpm 894 kB/s | 67 kB 00:00 2024-08-20T06:24:52.806 INFO:teuthology.orchestra.run.smithi183.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 13 MB/s | 839 kB 00:00 2024-08-20T06:24:52.831 INFO:teuthology.orchestra.run.smithi183.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T06:24:52.856 INFO:teuthology.orchestra.run.smithi183.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T06:24:52.864 INFO:teuthology.orchestra.run.smithi107.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 198 kB/s | 135 kB 00:00 2024-08-20T06:24:52.882 INFO:teuthology.orchestra.run.smithi183.stdout:(116/150): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-08-20T06:24:52.898 INFO:teuthology.orchestra.run.smithi107.stdout:(46/150): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-08-20T06:24:52.907 INFO:teuthology.orchestra.run.smithi183.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-08-20T06:24:52.932 INFO:teuthology.orchestra.run.smithi183.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-08-20T06:24:52.957 INFO:teuthology.orchestra.run.smithi183.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-20T06:24:52.983 INFO:teuthology.orchestra.run.smithi183.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-20T06:24:52.998 INFO:teuthology.orchestra.run.smithi107.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-08-20T06:24:53.008 INFO:teuthology.orchestra.run.smithi183.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-20T06:24:53.042 INFO:teuthology.orchestra.run.smithi183.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-20T06:24:53.076 INFO:teuthology.orchestra.run.smithi183.stdout:(123/150): protobuf-compiler-3.14.0-13.el9.x86_ 805 kB/s | 863 kB 00:01 2024-08-20T06:24:53.082 INFO:teuthology.orchestra.run.smithi107.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-08-20T06:24:53.092 INFO:teuthology.orchestra.run.smithi183.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-08-20T06:24:53.118 INFO:teuthology.orchestra.run.smithi183.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 1.3 MB/s | 23 kB 00:00 2024-08-20T06:24:53.124 INFO:teuthology.orchestra.run.smithi107.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-08-20T06:24:53.135 INFO:teuthology.orchestra.run.smithi183.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 1.3 MB/s | 19 kB 00:00 2024-08-20T06:24:53.152 INFO:teuthology.orchestra.run.smithi183.stdout:(127/150): python3-jaraco-classes-3.2.1-5.el9.n 233 kB/s | 18 kB 00:00 2024-08-20T06:24:53.158 INFO:teuthology.orchestra.run.smithi107.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 656 kB/s | 22 kB 00:00 2024-08-20T06:24:53.168 INFO:teuthology.orchestra.run.smithi183.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 590 kB/s | 20 kB 00:00 2024-08-20T06:24:53.185 INFO:teuthology.orchestra.run.smithi183.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 2.7 MB/s | 46 kB 00:00 2024-08-20T06:24:53.199 INFO:teuthology.orchestra.run.smithi107.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-08-20T06:24:53.211 INFO:teuthology.orchestra.run.smithi183.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-20T06:24:53.228 INFO:teuthology.orchestra.run.smithi183.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-08-20T06:24:53.233 INFO:teuthology.orchestra.run.smithi107.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 980 kB/s | 33 kB 00:00 2024-08-20T06:24:53.278 INFO:teuthology.orchestra.run.smithi183.stdout:(132/150): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-08-20T06:24:53.303 INFO:teuthology.orchestra.run.smithi183.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-08-20T06:24:53.320 INFO:teuthology.orchestra.run.smithi183.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 394 kB/s | 16 kB 00:00 2024-08-20T06:24:53.337 INFO:teuthology.orchestra.run.smithi183.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-08-20T06:24:53.362 INFO:teuthology.orchestra.run.smithi183.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 675 kB/s | 28 kB 00:00 2024-08-20T06:24:53.383 INFO:teuthology.orchestra.run.smithi107.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-08-20T06:24:53.388 INFO:teuthology.orchestra.run.smithi183.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-08-20T06:24:53.409 INFO:teuthology.orchestra.run.smithi107.stdout:(54/150): python3-requests-2.25.1-8.el9.noarch. 170 kB/s | 125 kB 00:00 2024-08-20T06:24:53.413 INFO:teuthology.orchestra.run.smithi183.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-20T06:24:53.426 INFO:teuthology.orchestra.run.smithi107.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 434 kB/s | 18 kB 00:00 2024-08-20T06:24:53.447 INFO:teuthology.orchestra.run.smithi183.stdout:(139/150): python3-kubernetes-26.1.0-2.el9.noar 3.5 MB/s | 1.0 MB 00:00 2024-08-20T06:24:53.467 INFO:teuthology.orchestra.run.smithi107.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-08-20T06:24:53.472 INFO:teuthology.orchestra.run.smithi183.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-08-20T06:24:53.484 INFO:teuthology.orchestra.run.smithi107.stdout:(57/150): libaio-devel-0.3.111-13.el9.x86_64.rp 148 kB/s | 11 kB 00:00 2024-08-20T06:24:53.497 INFO:teuthology.orchestra.run.smithi183.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 427 kB/s | 36 kB 00:00 2024-08-20T06:24:53.510 INFO:teuthology.orchestra.run.smithi107.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-08-20T06:24:53.522 INFO:teuthology.orchestra.run.smithi183.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-20T06:24:53.547 INFO:teuthology.orchestra.run.smithi183.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-20T06:24:53.552 INFO:teuthology.orchestra.run.smithi107.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 14 MB/s | 581 kB 00:00 2024-08-20T06:24:53.564 INFO:teuthology.orchestra.run.smithi183.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-08-20T06:24:53.589 INFO:teuthology.orchestra.run.smithi183.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 479 kB/s | 20 kB 00:00 2024-08-20T06:24:53.594 INFO:teuthology.orchestra.run.smithi107.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 610 kB/s | 25 kB 00:00 2024-08-20T06:24:53.615 INFO:teuthology.orchestra.run.smithi183.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-20T06:24:53.636 INFO:teuthology.orchestra.run.smithi107.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-08-20T06:24:53.640 INFO:teuthology.orchestra.run.smithi183.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-08-20T06:24:53.661 INFO:teuthology.orchestra.run.smithi107.stdout:(62/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-08-20T06:24:53.682 INFO:teuthology.orchestra.run.smithi183.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-08-20T06:24:53.686 INFO:teuthology.orchestra.run.smithi107.stdout:(63/150): python3-urllib3-1.26.5-6.el9.noarch.r 261 kB/s | 215 kB 00:00 2024-08-20T06:24:53.703 INFO:teuthology.orchestra.run.smithi107.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 4.4 MB/s | 188 kB 00:00 2024-08-20T06:24:53.728 INFO:teuthology.orchestra.run.smithi107.stdout:(65/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-08-20T06:24:53.829 INFO:teuthology.orchestra.run.smithi107.stdout:(66/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-08-20T06:24:53.854 INFO:teuthology.orchestra.run.smithi107.stdout:(67/150): openblas-0.3.26-1.el9.x86_64.rpm 229 kB/s | 38 kB 00:00 2024-08-20T06:24:53.899 INFO:teuthology.orchestra.run.smithi183.stdout:(149/150): librbd1-19.0.0-5872.gef427541.el9.x8 13 MB/s | 3.2 MB 00:00 2024-08-20T06:24:53.921 INFO:teuthology.orchestra.run.smithi107.stdout:(68/150): openblas-openmp-0.3.26-1.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-08-20T06:24:53.949 INFO:teuthology.orchestra.run.smithi183.stdout:(150/150): librados2-19.0.0-5872.gef427541.el9. 10 MB/s | 3.5 MB 00:00 2024-08-20T06:24:53.955 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:24:53.955 INFO:teuthology.orchestra.run.smithi183.stdout:Total 25 MB/s | 232 MB 00:09 2024-08-20T06:24:53.963 INFO:teuthology.orchestra.run.smithi107.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-08-20T06:24:53.988 INFO:teuthology.orchestra.run.smithi107.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-08-20T06:24:54.047 INFO:teuthology.orchestra.run.smithi107.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-08-20T06:24:54.081 INFO:teuthology.orchestra.run.smithi107.stdout:(72/150): postgresql-contrib-13.16-1.el9.x86_64 3.9 MB/s | 910 kB 00:00 2024-08-20T06:24:54.098 INFO:teuthology.orchestra.run.smithi107.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-08-20T06:24:54.165 INFO:teuthology.orchestra.run.smithi107.stdout:(74/150): postgresql-server-13.16-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-08-20T06:24:54.190 INFO:teuthology.orchestra.run.smithi107.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 668 kB/s | 61 kB 00:00 2024-08-20T06:24:54.203 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 18 kB 00:00 2024-08-20T06:24:54.215 INFO:teuthology.orchestra.run.smithi107.stdout:(76/150): python3-devel-3.9.19-7.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-08-20T06:24:54.241 INFO:teuthology.orchestra.run.smithi107.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-20T06:24:54.266 INFO:teuthology.orchestra.run.smithi107.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-08-20T06:24:54.291 INFO:teuthology.orchestra.run.smithi107.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 387 kB/s | 19 kB 00:00 2024-08-20T06:24:54.316 INFO:teuthology.orchestra.run.smithi107.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-20T06:24:54.341 INFO:teuthology.orchestra.run.smithi107.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-08-20T06:24:54.386 INFO:teuthology.orchestra.run.smithi107.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-08-20T06:24:54.459 INFO:teuthology.orchestra.run.smithi107.stdout:(83/150): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-08-20T06:24:54.484 INFO:teuthology.orchestra.run.smithi107.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-08-20T06:24:54.509 INFO:teuthology.orchestra.run.smithi107.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-20T06:24:54.526 INFO:teuthology.orchestra.run.smithi107.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-20T06:24:54.593 INFO:teuthology.orchestra.run.smithi107.stdout:(87/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 18 MB/s | 5.1 MB 00:00 2024-08-20T06:24:54.618 INFO:teuthology.orchestra.run.smithi107.stdout:(88/150): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-08-20T06:24:54.644 INFO:teuthology.orchestra.run.smithi107.stdout:(89/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-08-20T06:24:54.660 INFO:teuthology.orchestra.run.smithi107.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-20T06:24:54.678 INFO:teuthology.orchestra.run.smithi107.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-08-20T06:24:54.703 INFO:teuthology.orchestra.run.smithi107.stdout:(92/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-08-20T06:24:54.720 INFO:teuthology.orchestra.run.smithi107.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 994 kB/s | 42 kB 00:00 2024-08-20T06:24:54.737 INFO:teuthology.orchestra.run.smithi107.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-08-20T06:24:54.787 INFO:teuthology.orchestra.run.smithi107.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-08-20T06:24:54.812 INFO:teuthology.orchestra.run.smithi107.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 478 kB/s | 36 kB 00:00 2024-08-20T06:24:54.838 INFO:teuthology.orchestra.run.smithi107.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-08-20T06:24:54.863 INFO:teuthology.orchestra.run.smithi107.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-08-20T06:24:54.888 INFO:teuthology.orchestra.run.smithi107.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-08-20T06:24:54.914 INFO:teuthology.orchestra.run.smithi107.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-20T06:24:54.939 INFO:teuthology.orchestra.run.smithi107.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-20T06:24:54.957 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 35 MB/s | 22 MB 00:00 2024-08-20T06:24:54.964 INFO:teuthology.orchestra.run.smithi107.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-08-20T06:24:55.049 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-08-20T06:24:55.176 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-08-20T06:24:55.176 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-08-20T06:24:55.215 INFO:teuthology.orchestra.run.smithi107.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-08-20T06:24:55.265 INFO:teuthology.orchestra.run.smithi107.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-20T06:24:55.298 INFO:teuthology.orchestra.run.smithi107.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-20T06:24:55.341 INFO:teuthology.orchestra.run.smithi107.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 732 kB/s | 19 kB 00:00 2024-08-20T06:24:55.441 INFO:teuthology.orchestra.run.smithi107.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 44 MB/s | 4.4 MB 00:00 2024-08-20T06:24:55.466 INFO:teuthology.orchestra.run.smithi107.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-20T06:24:55.491 INFO:teuthology.orchestra.run.smithi107.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-20T06:24:55.508 INFO:teuthology.orchestra.run.smithi107.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 3.9 MB/s | 67 kB 00:00 2024-08-20T06:24:55.534 INFO:teuthology.orchestra.run.smithi107.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-20T06:24:55.567 INFO:teuthology.orchestra.run.smithi107.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 25 MB/s | 839 kB 00:00 2024-08-20T06:24:55.601 INFO:teuthology.orchestra.run.smithi107.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-08-20T06:24:55.626 INFO:teuthology.orchestra.run.smithi107.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T06:24:55.651 INFO:teuthology.orchestra.run.smithi107.stdout:(115/150): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-08-20T06:24:55.677 INFO:teuthology.orchestra.run.smithi107.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 654 kB/s | 32 kB 00:00 2024-08-20T06:24:55.710 INFO:teuthology.orchestra.run.smithi107.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.4 MB/s | 172 kB 00:00 2024-08-20T06:24:55.736 INFO:teuthology.orchestra.run.smithi107.stdout:(118/150): python3-certifi-2023.05.07-4.el9.noa 168 kB/s | 14 kB 00:00 2024-08-20T06:24:55.752 INFO:teuthology.orchestra.run.smithi107.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 8.7 MB/s | 358 kB 00:00 2024-08-20T06:24:55.780 INFO:teuthology.orchestra.run.smithi107.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.7 MB/s | 303 kB 00:00 2024-08-20T06:24:55.805 INFO:teuthology.orchestra.run.smithi107.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.1 MB/s | 220 kB 00:00 2024-08-20T06:24:55.830 INFO:teuthology.orchestra.run.smithi107.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-20T06:24:55.864 INFO:teuthology.orchestra.run.smithi107.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 320 kB/s | 11 kB 00:00 2024-08-20T06:24:55.881 INFO:teuthology.orchestra.run.smithi107.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-08-20T06:24:55.906 INFO:teuthology.orchestra.run.smithi107.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-08-20T06:24:55.932 INFO:teuthology.orchestra.run.smithi107.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-08-20T06:24:55.957 INFO:teuthology.orchestra.run.smithi107.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-08-20T06:24:56.007 INFO:teuthology.orchestra.run.smithi107.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 21 MB/s | 1.0 MB 00:00 2024-08-20T06:24:56.049 INFO:teuthology.orchestra.run.smithi107.stdout:(129/150): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-08-20T06:24:56.074 INFO:teuthology.orchestra.run.smithi107.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 691 kB/s | 46 kB 00:00 2024-08-20T06:24:56.100 INFO:teuthology.orchestra.run.smithi107.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-08-20T06:24:56.125 INFO:teuthology.orchestra.run.smithi107.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-08-20T06:24:56.159 INFO:teuthology.orchestra.run.smithi107.stdout:(133/150): protobuf-compiler-3.14.0-13.el9.x86_ 723 kB/s | 863 kB 00:01 2024-08-20T06:24:56.175 INFO:teuthology.orchestra.run.smithi107.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-08-20T06:24:56.201 INFO:teuthology.orchestra.run.smithi107.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.6 MB/s | 28 kB 00:00 2024-08-20T06:24:56.226 INFO:teuthology.orchestra.run.smithi107.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 1.3 MB/s | 31 kB 00:00 2024-08-20T06:24:56.251 INFO:teuthology.orchestra.run.smithi107.stdout:(137/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-08-20T06:24:56.277 INFO:teuthology.orchestra.run.smithi107.stdout:(138/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-08-20T06:24:56.293 INFO:teuthology.orchestra.run.smithi107.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-08-20T06:24:56.310 INFO:teuthology.orchestra.run.smithi107.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-08-20T06:24:56.336 INFO:teuthology.orchestra.run.smithi107.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-20T06:24:56.353 INFO:teuthology.orchestra.run.smithi107.stdout:(142/150): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-08-20T06:24:56.378 INFO:teuthology.orchestra.run.smithi107.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-08-20T06:24:56.403 INFO:teuthology.orchestra.run.smithi107.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-08-20T06:24:56.420 INFO:teuthology.orchestra.run.smithi107.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-08-20T06:24:56.446 INFO:teuthology.orchestra.run.smithi107.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 4.5 MB/s | 192 kB 00:00 2024-08-20T06:24:56.470 INFO:teuthology.orchestra.run.smithi107.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-08-20T06:24:56.521 INFO:teuthology.orchestra.run.smithi107.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-20T06:24:56.705 INFO:teuthology.orchestra.run.smithi107.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 13 MB/s | 3.5 MB 00:00 2024-08-20T06:24:56.779 INFO:teuthology.orchestra.run.smithi107.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 10 MB/s | 3.2 MB 00:00 2024-08-20T06:24:56.785 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:24:56.785 INFO:teuthology.orchestra.run.smithi107.stdout:Total 19 MB/s | 232 MB 00:11 2024-08-20T06:24:57.377 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-08-20T06:24:57.378 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-08-20T06:24:57.862 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-08-20T06:24:57.991 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-08-20T06:24:57.991 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-08-20T06:25:00.174 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-08-20T06:25:00.174 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-08-20T06:25:00.321 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-08-20T06:25:02.500 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:25:02.501 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T06:25:02.512 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T06:25:02.512 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T06:25:02.513 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T06:25:02.514 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T06:25:02.516 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T06:25:02.518 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T06:25:02.519 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T06:25:02.579 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-08-20T06:25:02.587 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-08-20T06:25:02.587 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-08-20T06:25:02.587 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-08-20T06:25:02.587 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-08-20T06:25:02.587 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T06:25:02.588 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T06:25:02.589 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T06:25:02.590 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T06:25:02.591 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T06:25:02.592 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T06:25:02.593 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T06:25:02.594 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T06:25:02.595 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T06:25:02.596 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T06:25:02.597 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T06:25:02.598 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T06:25:02.599 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-08-20T06:25:02.600 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:02.601 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 232 M 2024-08-20T06:25:02.601 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-08-20T06:25:03.326 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-08-20T06:25:03.347 INFO:teuthology.orchestra.run.smithi193.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-08-20T06:25:03.401 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T06:25:03.426 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T06:25:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T06:25:03.531 INFO:teuthology.orchestra.run.smithi193.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 4.8 MB/s | 901 kB 00:00 2024-08-20T06:25:03.623 INFO:teuthology.orchestra.run.smithi193.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 1.5 MB/s | 145 kB 00:00 2024-08-20T06:25:03.698 INFO:teuthology.orchestra.run.smithi193.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-20T06:25:03.799 INFO:teuthology.orchestra.run.smithi193.stdout:(5/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 15 MB/s | 1.5 MB 00:00 2024-08-20T06:25:03.837 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T06:25:03.849 INFO:teuthology.orchestra.run.smithi193.stdout:(6/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 10 MB/s | 2.4 MB 00:00 2024-08-20T06:25:03.952 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:04.079 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:04.091 INFO:teuthology.orchestra.run.smithi193.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 16 MB/s | 4.8 MB 00:00 2024-08-20T06:25:04.103 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:04.182 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:04.220 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:25:04.249 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T06:25:04.275 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T06:25:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T06:25:04.525 INFO:teuthology.orchestra.run.smithi193.stdout:(8/150): ceph-common-19.0.0-5872.gef427541.el9. 16 MB/s | 22 MB 00:01 2024-08-20T06:25:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T06:25:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:04.583 INFO:teuthology.orchestra.run.smithi193.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 432 kB/s | 25 kB 00:00 2024-08-20T06:25:04.671 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:04.701 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:04.761 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:04.785 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:04.833 INFO:teuthology.orchestra.run.smithi193.stdout:(10/150): ceph-radosgw-19.0.0-5872.gef427541.el 15 MB/s | 11 MB 00:00 2024-08-20T06:25:05.025 INFO:teuthology.orchestra.run.smithi193.stdout:(11/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 15 MB/s | 17 MB 00:01 2024-08-20T06:25:05.050 INFO:teuthology.orchestra.run.smithi193.stdout:(12/150): cephfs-mirror-19.0.0-5872.gef427541.e 1.0 MB/s | 225 kB 00:00 2024-08-20T06:25:05.112 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:05.134 INFO:teuthology.orchestra.run.smithi193.stdout:(13/150): cephfs-shell-19.0.0-5872.gef427541.el 249 kB/s | 27 kB 00:00 2024-08-20T06:25:05.144 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:05.151 INFO:teuthology.orchestra.run.smithi193.stdout:(14/150): libcephfs-devel-19.0.0-5872.gef427541 316 kB/s | 32 kB 00:00 2024-08-20T06:25:05.202 INFO:teuthology.orchestra.run.smithi193.stdout:(15/150): libcephsqlite-19.0.0-5872.gef427541.e 3.2 MB/s | 163 kB 00:00 2024-08-20T06:25:05.227 INFO:teuthology.orchestra.run.smithi193.stdout:(16/150): libcephfs2-19.0.0-5872.gef427541.el9. 8.0 MB/s | 755 kB 00:00 2024-08-20T06:25:05.252 INFO:teuthology.orchestra.run.smithi193.stdout:(17/150): librados-devel-19.0.0-5872.gef427541. 2.5 MB/s | 126 kB 00:00 2024-08-20T06:25:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T06:25:05.286 INFO:teuthology.orchestra.run.smithi193.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 8.5 MB/s | 508 kB 00:00 2024-08-20T06:25:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T06:25:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T06:25:05.344 INFO:teuthology.orchestra.run.smithi193.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 774 kB/s | 45 kB 00:00 2024-08-20T06:25:05.403 INFO:teuthology.orchestra.run.smithi193.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 2.3 MB/s | 138 kB 00:00 2024-08-20T06:25:05.409 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T06:25:05.462 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T06:25:05.462 INFO:teuthology.orchestra.run.smithi193.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 3.1 MB/s | 188 kB 00:00 2024-08-20T06:25:05.502 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T06:25:05.528 INFO:teuthology.orchestra.run.smithi193.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 5.5 MB/s | 371 kB 00:00 2024-08-20T06:25:05.545 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T06:25:05.591 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T06:25:05.604 INFO:teuthology.orchestra.run.smithi193.stdout:(23/150): librgw2-19.0.0-5872.gef427541.el9.x86 16 MB/s | 5.5 MB 00:00 2024-08-20T06:25:05.631 INFO:teuthology.orchestra.run.smithi193.stdout:(24/150): python3-rbd-19.0.0-5872.gef427541.el9 3.5 MB/s | 355 kB 00:00 2024-08-20T06:25:05.654 INFO:teuthology.orchestra.run.smithi193.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 2.3 MB/s | 116 kB 00:00 2024-08-20T06:25:05.680 INFO:teuthology.orchestra.run.smithi193.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-20T06:25:05.722 INFO:teuthology.orchestra.run.smithi193.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 4.0 MB/s | 170 kB 00:00 2024-08-20T06:25:05.780 INFO:teuthology.orchestra.run.smithi193.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 591 kB/s | 34 kB 00:00 2024-08-20T06:25:05.839 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T06:25:05.840 INFO:teuthology.orchestra.run.smithi193.stdout:(29/150): rbd-mirror-19.0.0-5872.gef427541.el9. 17 MB/s | 3.2 MB 00:00 2024-08-20T06:25:05.856 INFO:teuthology.orchestra.run.smithi193.stdout:(30/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 2.0 MB/s | 150 kB 00:00 2024-08-20T06:25:05.909 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T06:25:05.970 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T06:25:06.008 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T06:25:06.044 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T06:25:06.084 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-08-20T06:25:06.121 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T06:25:06.148 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T06:25:06.153 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T06:25:06.173 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T06:25:06.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T06:25:06.226 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T06:25:06.255 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T06:25:06.298 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T06:25:06.341 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T06:25:06.364 INFO:teuthology.orchestra.run.smithi193.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-08-20T06:25:06.418 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T06:25:06.431 INFO:teuthology.orchestra.run.smithi193.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 4.2 MB/s | 284 kB 00:00 2024-08-20T06:25:06.492 INFO:teuthology.orchestra.run.smithi193.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 818 kB/s | 49 kB 00:00 2024-08-20T06:25:06.543 INFO:teuthology.orchestra.run.smithi193.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 330 kB/s | 16 kB 00:00 2024-08-20T06:25:06.551 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T06:25:06.595 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T06:25:06.618 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T06:25:06.676 INFO:teuthology.orchestra.run.smithi193.stdout:(35/150): ceph-mgr-dashboard-19.0.0-5872.gef427 17 MB/s | 14 MB 00:00 2024-08-20T06:25:06.701 INFO:teuthology.orchestra.run.smithi193.stdout:(36/150): ceph-volume-19.0.0-5872.gef427541.el9 1.8 MB/s | 285 kB 00:00 2024-08-20T06:25:06.732 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:06.742 INFO:teuthology.orchestra.run.smithi193.stdout:(37/150): cephadm-19.0.0-5872.gef427541.el9.noa 11 MB/s | 772 kB 00:00 2024-08-20T06:25:06.759 INFO:teuthology.orchestra.run.smithi193.stdout:(38/150): cephfs-top-19.0.0-5872.gef427541.el9. 316 kB/s | 18 kB 00:00 2024-08-20T06:25:06.854 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:06.876 INFO:teuthology.orchestra.run.smithi193.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-08-20T06:25:06.883 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:06.943 INFO:teuthology.orchestra.run.smithi193.stdout:(40/150): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-08-20T06:25:06.994 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:07.033 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:25:07.063 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T06:25:07.088 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T06:25:07.153 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T06:25:07.239 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T06:25:07.267 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T06:25:07.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T06:25:07.319 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T06:25:07.352 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T06:25:07.356 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T06:25:07.377 INFO:teuthology.orchestra.run.smithi193.stdout:(41/150): ceph-test-19.0.0-5872.gef427541.el9.x 18 MB/s | 50 MB 00:02 2024-08-20T06:25:07.380 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:07.389 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T06:25:07.402 INFO:teuthology.orchestra.run.smithi193.stdout:(42/150): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-08-20T06:25:07.428 INFO:teuthology.orchestra.run.smithi193.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 523 kB/s | 253 kB 00:00 2024-08-20T06:25:07.453 INFO:teuthology.orchestra.run.smithi193.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-08-20T06:25:07.484 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T06:25:07.484 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:07.486 INFO:teuthology.orchestra.run.smithi193.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-08-20T06:25:07.506 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:07.520 INFO:teuthology.orchestra.run.smithi193.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-08-20T06:25:07.545 INFO:teuthology.orchestra.run.smithi193.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-20T06:25:07.546 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T06:25:07.570 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:07.598 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:07.604 INFO:teuthology.orchestra.run.smithi193.stdout:(48/150): python3-cryptography-36.0.1-4.el9.x86 5.4 MB/s | 1.2 MB 00:00 2024-08-20T06:25:07.619 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T06:25:07.621 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:07.658 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T06:25:07.694 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T06:25:07.742 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T06:25:07.768 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T06:25:07.811 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T06:25:07.853 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T06:25:07.877 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:07.907 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T06:25:08.013 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T06:25:08.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T06:25:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T06:25:08.073 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T06:25:08.080 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T06:25:08.140 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T06:25:08.184 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T06:25:08.192 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T06:25:08.214 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T06:25:08.232 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T06:25:08.254 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T06:25:08.275 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T06:25:08.287 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T06:25:08.309 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T06:25:08.311 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T06:25:08.336 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T06:25:08.371 INFO:teuthology.orchestra.run.smithi193.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 139 kB/s | 107 kB 00:00 2024-08-20T06:25:08.406 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T06:25:08.406 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T06:25:08.459 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T06:25:08.489 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T06:25:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T06:25:08.540 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T06:25:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T06:25:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T06:25:08.619 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T06:25:08.639 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T06:25:08.652 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T06:25:08.699 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T06:25:08.724 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T06:25:08.795 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T06:25:08.810 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T06:25:08.839 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T06:25:08.846 INFO:teuthology.orchestra.run.smithi193.stdout:(50/150): automake-1.16.2-8.el9.noarch.rpm 520 kB/s | 689 kB 00:01 2024-08-20T06:25:08.896 INFO:teuthology.orchestra.run.smithi193.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 41 kB/s | 22 kB 00:00 2024-08-20T06:25:08.914 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T06:25:08.994 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T06:25:09.028 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T06:25:09.050 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T06:25:09.058 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T06:25:09.084 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T06:25:09.099 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T06:25:09.134 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T06:25:09.185 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T06:25:09.218 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T06:25:09.280 INFO:teuthology.orchestra.run.smithi193.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 544 kB/s | 943 kB 00:01 2024-08-20T06:25:09.308 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T06:25:09.357 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T06:25:09.387 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T06:25:09.464 INFO:teuthology.orchestra.run.smithi193.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-08-20T06:25:09.484 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T06:25:09.539 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T06:25:09.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T06:25:09.720 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T06:25:09.748 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T06:25:09.780 INFO:teuthology.orchestra.run.smithi193.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 336 kB/s | 313 kB 00:00 2024-08-20T06:25:09.784 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T06:25:09.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T06:25:09.846 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T06:25:09.938 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T06:25:09.947 INFO:teuthology.orchestra.run.smithi193.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-08-20T06:25:10.027 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T06:25:10.100 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T06:25:10.131 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T06:25:10.167 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T06:25:10.214 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T06:25:10.250 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T06:25:10.273 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T06:25:10.289 INFO:teuthology.orchestra.run.smithi193.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 22 kB/s | 11 kB 00:00 2024-08-20T06:25:10.293 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T06:25:10.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T06:25:10.335 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T06:25:10.348 INFO:teuthology.orchestra.run.smithi193.stdout:(57/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-08-20T06:25:10.361 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T06:25:10.465 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T06:25:10.472 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T06:25:10.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T06:25:10.516 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T06:25:10.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T06:25:10.560 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T06:25:10.651 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T06:25:10.657 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T06:25:10.689 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T06:25:10.689 INFO:teuthology.orchestra.run.smithi193.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-08-20T06:25:10.696 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T06:25:10.737 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T06:25:10.769 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T06:25:10.785 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T06:25:10.865 INFO:teuthology.orchestra.run.smithi193.stdout:(59/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-08-20T06:25:10.873 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T06:25:10.887 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T06:25:10.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T06:25:10.981 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T06:25:11.021 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T06:25:11.059 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T06:25:11.091 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T06:25:11.122 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T06:25:11.166 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T06:25:11.203 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T06:25:11.215 INFO:teuthology.orchestra.run.smithi193.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 764 kB/s | 662 kB 00:00 2024-08-20T06:25:11.294 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T06:25:11.407 INFO:teuthology.orchestra.run.smithi193.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 47 kB/s | 25 kB 00:00 2024-08-20T06:25:11.453 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T06:25:11.482 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T06:25:11.516 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T06:25:11.541 INFO:teuthology.orchestra.run.smithi193.stdout:(62/150): libtool-2.4.6-46.el9.x86_64.rpm 683 kB/s | 581 kB 00:00 2024-08-20T06:25:11.558 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T06:25:11.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T06:25:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T06:25:11.680 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T06:25:11.722 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T06:25:11.753 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T06:25:11.778 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T06:25:11.791 INFO:teuthology.orchestra.run.smithi193.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-08-20T06:25:11.858 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T06:25:11.912 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T06:25:12.224 INFO:teuthology.orchestra.run.smithi193.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 358 kB/s | 292 kB 00:00 2024-08-20T06:25:12.226 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:12.266 INFO:teuthology.orchestra.run.smithi193.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 260 kB/s | 188 kB 00:00 2024-08-20T06:25:12.522 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:12.592 INFO:teuthology.orchestra.run.smithi193.stdout:(66/150): openblas-0.3.26-1.el9.x86_64.rpm 48 kB/s | 38 kB 00:00 2024-08-20T06:25:12.643 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T06:25:12.668 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:12.692 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T06:25:12.726 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T06:25:12.826 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T06:25:12.873 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T06:25:12.918 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T06:25:12.962 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:13.008 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T06:25:13.027 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:13.055 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T06:25:13.087 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T06:25:13.141 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:25:13.175 INFO:teuthology.orchestra.run.smithi193.stdout:(67/150): openblas-openmp-0.3.26-1.el9.x86_64.r 5.2 MB/s | 4.9 MB 00:00 2024-08-20T06:25:13.181 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T06:25:13.211 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T06:25:13.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T06:25:13.301 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T06:25:13.321 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:13.384 INFO:teuthology.orchestra.run.smithi193.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 2.0 MB/s | 1.6 MB 00:00 2024-08-20T06:25:13.386 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T06:25:13.417 INFO:teuthology.orchestra.run.smithi193.stdout:(69/150): postgresql-private-libs-13.16-1.el9.x 4.0 MB/s | 137 kB 00:00 2024-08-20T06:25:13.526 INFO:teuthology.orchestra.run.smithi193.stdout:(70/150): postgresql-server-13.16-1.el9.x86_64. 53 MB/s | 5.8 MB 00:00 2024-08-20T06:25:13.528 INFO:teuthology.orchestra.run.smithi193.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://centos-stream-distro.1gservers.com/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 104.251.122.5) 2024-08-20T06:25:13.538 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T06:25:13.574 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:13.576 INFO:teuthology.orchestra.run.smithi193.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-08-20T06:25:13.618 INFO:teuthology.orchestra.run.smithi193.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-08-20T06:25:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:13.694 INFO:teuthology.orchestra.run.smithi193.stdout:(73/150): postgresql-contrib-13.16-1.el9.x86_64 1.7 MB/s | 910 kB 00:00 2024-08-20T06:25:13.694 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T06:25:13.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T06:25:13.777 INFO:teuthology.orchestra.run.smithi193.stdout:(74/150): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-08-20T06:25:13.794 INFO:teuthology.orchestra.run.smithi193.stdout:(75/150): python3-colorama-0.4.6-3.el9.noarch.r 612 kB/s | 61 kB 00:00 2024-08-20T06:25:13.819 INFO:teuthology.orchestra.run.smithi193.stdout:(76/150): python3-devel-3.9.19-7.el9.x86_64.rpm 5.8 MB/s | 245 kB 00:00 2024-08-20T06:25:13.845 INFO:teuthology.orchestra.run.smithi193.stdout:(77/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-08-20T06:25:13.861 INFO:teuthology.orchestra.run.smithi193.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-08-20T06:25:13.878 INFO:teuthology.orchestra.run.smithi193.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 573 kB/s | 19 kB 00:00 2024-08-20T06:25:13.904 INFO:teuthology.orchestra.run.smithi193.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-08-20T06:25:13.919 INFO:teuthology.orchestra.run.smithi193.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 867 kB/s | 35 kB 00:00 2024-08-20T06:25:14.011 INFO:teuthology.orchestra.run.smithi193.stdout:(82/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 48 MB/s | 5.1 MB 00:00 2024-08-20T06:25:14.036 INFO:teuthology.orchestra.run.smithi193.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-20T06:25:14.061 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T06:25:14.061 INFO:teuthology.orchestra.run.smithi193.stdout:(84/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-08-20T06:25:14.078 INFO:teuthology.orchestra.run.smithi193.stdout:(85/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-08-20T06:25:14.095 INFO:teuthology.orchestra.run.smithi193.stdout:(86/150): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-08-20T06:25:14.129 INFO:teuthology.orchestra.run.smithi193.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-08-20T06:25:14.154 INFO:teuthology.orchestra.run.smithi193.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-20T06:25:14.167 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T06:25:14.180 INFO:teuthology.orchestra.run.smithi193.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-08-20T06:25:14.205 INFO:teuthology.orchestra.run.smithi193.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-20T06:25:14.221 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T06:25:14.243 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:14.247 INFO:teuthology.orchestra.run.smithi193.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 803 kB/s | 54 kB 00:00 2024-08-20T06:25:14.315 INFO:teuthology.orchestra.run.smithi193.stdout:(92/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-08-20T06:25:14.339 INFO:teuthology.orchestra.run.smithi193.stdout:(93/150): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-08-20T06:25:14.390 INFO:teuthology.orchestra.run.smithi193.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-08-20T06:25:14.415 INFO:teuthology.orchestra.run.smithi193.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-08-20T06:25:14.440 INFO:teuthology.orchestra.run.smithi193.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-08-20T06:25:14.466 INFO:teuthology.orchestra.run.smithi193.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-08-20T06:25:14.507 INFO:teuthology.orchestra.run.smithi193.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-08-20T06:25:14.525 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:14.533 INFO:teuthology.orchestra.run.smithi193.stdout:(99/150): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-08-20T06:25:14.553 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T06:25:14.558 INFO:teuthology.orchestra.run.smithi193.stdout:(100/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-08-20T06:25:14.583 INFO:teuthology.orchestra.run.smithi193.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-20T06:25:14.608 INFO:teuthology.orchestra.run.smithi193.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-08-20T06:25:14.634 INFO:teuthology.orchestra.run.smithi193.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-08-20T06:25:14.684 INFO:teuthology.orchestra.run.smithi193.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-20T06:25:14.718 INFO:teuthology.orchestra.run.smithi193.stdout:(105/150): protobuf-compiler-3.14.0-13.el9.x86_ 7.7 MB/s | 863 kB 00:00 2024-08-20T06:25:14.751 INFO:teuthology.orchestra.run.smithi193.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-08-20T06:25:14.785 INFO:teuthology.orchestra.run.smithi193.stdout:(107/150): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-08-20T06:25:14.810 INFO:teuthology.orchestra.run.smithi193.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-20T06:25:14.969 INFO:teuthology.orchestra.run.smithi193.stdout:(109/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-08-20T06:25:14.994 INFO:teuthology.orchestra.run.smithi193.stdout:(110/150): liboath-2.6.7-2.el9.x86_64.rpm 265 kB/s | 49 kB 00:00 2024-08-20T06:25:15.036 INFO:teuthology.orchestra.run.smithi193.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.3 MB/s | 151 kB 00:00 2024-08-20T06:25:15.061 INFO:teuthology.orchestra.run.smithi193.stdout:(112/150): libunwind-1.6.2-1.el9.x86_64.rpm 733 kB/s | 67 kB 00:00 2024-08-20T06:25:15.145 INFO:teuthology.orchestra.run.smithi193.stdout:(113/150): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-08-20T06:25:15.170 INFO:teuthology.orchestra.run.smithi193.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T06:25:15.195 INFO:teuthology.orchestra.run.smithi193.stdout:(115/150): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-08-20T06:25:15.221 INFO:teuthology.orchestra.run.smithi193.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-08-20T06:25:15.246 INFO:teuthology.orchestra.run.smithi193.stdout:(117/150): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-08-20T06:25:15.280 INFO:teuthology.orchestra.run.smithi193.stdout:(118/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.4 MB/s | 839 kB 00:00 2024-08-20T06:25:15.305 INFO:teuthology.orchestra.run.smithi193.stdout:(119/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-08-20T06:25:15.330 INFO:teuthology.orchestra.run.smithi193.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-08-20T06:25:15.356 INFO:teuthology.orchestra.run.smithi193.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-08-20T06:25:15.381 INFO:teuthology.orchestra.run.smithi193.stdout:(122/150): python3-google-auth-2.33.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-08-20T06:25:15.398 INFO:teuthology.orchestra.run.smithi193.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 638 kB/s | 11 kB 00:00 2024-08-20T06:25:15.423 INFO:teuthology.orchestra.run.smithi193.stdout:(124/150): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-08-20T06:25:15.437 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T06:25:15.440 INFO:teuthology.orchestra.run.smithi193.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 423 kB/s | 18 kB 00:00 2024-08-20T06:25:15.457 INFO:teuthology.orchestra.run.smithi193.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 692 kB/s | 23 kB 00:00 2024-08-20T06:25:15.474 INFO:teuthology.orchestra.run.smithi193.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 583 kB/s | 19 kB 00:00 2024-08-20T06:25:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 588 kB/s | 20 kB 00:00 2024-08-20T06:25:15.508 INFO:teuthology.orchestra.run.smithi193.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-08-20T06:25:15.513 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:25:15.533 INFO:teuthology.orchestra.run.smithi193.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-20T06:25:15.547 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T06:25:15.550 INFO:teuthology.orchestra.run.smithi193.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-08-20T06:25:15.582 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T06:25:15.584 INFO:teuthology.orchestra.run.smithi193.stdout:(132/150): python3-kubernetes-26.1.0-2.el9.noar 9.3 MB/s | 1.0 MB 00:00 2024-08-20T06:25:15.609 INFO:teuthology.orchestra.run.smithi193.stdout:(133/150): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-08-20T06:25:15.634 INFO:teuthology.orchestra.run.smithi193.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-08-20T06:25:15.660 INFO:teuthology.orchestra.run.smithi193.stdout:(135/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-08-20T06:25:15.685 INFO:teuthology.orchestra.run.smithi193.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-08-20T06:25:15.710 INFO:teuthology.orchestra.run.smithi193.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-20T06:25:15.735 INFO:teuthology.orchestra.run.smithi193.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-20T06:25:15.760 INFO:teuthology.orchestra.run.smithi193.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-20T06:25:15.786 INFO:teuthology.orchestra.run.smithi193.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-08-20T06:25:15.803 INFO:teuthology.orchestra.run.smithi193.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-08-20T06:25:15.828 INFO:teuthology.orchestra.run.smithi193.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-20T06:25:15.870 INFO:teuthology.orchestra.run.smithi193.stdout:(143/150): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-08-20T06:25:15.895 INFO:teuthology.orchestra.run.smithi193.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-08-20T06:25:15.901 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T06:25:15.920 INFO:teuthology.orchestra.run.smithi193.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 802 kB/s | 20 kB 00:00 2024-08-20T06:25:15.945 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:15.946 INFO:teuthology.orchestra.run.smithi193.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-20T06:25:15.979 INFO:teuthology.orchestra.run.smithi193.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-08-20T06:25:16.030 INFO:teuthology.orchestra.run.smithi193.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-08-20T06:25:16.040 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:16.066 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T06:25:16.103 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T06:25:16.272 INFO:teuthology.orchestra.run.smithi193.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 11 MB/s | 3.5 MB 00:00 2024-08-20T06:25:16.330 INFO:teuthology.orchestra.run.smithi193.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 9.2 MB/s | 3.2 MB 00:00 2024-08-20T06:25:16.336 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:25:16.336 INFO:teuthology.orchestra.run.smithi193.stdout:Total 17 MB/s | 232 MB 00:13 2024-08-20T06:25:16.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T06:25:16.698 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:16.770 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:16.799 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T06:25:16.902 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:16.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:17.408 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-08-20T06:25:17.485 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:17.519 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:17.536 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-08-20T06:25:17.536 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-08-20T06:25:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:17.607 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T06:25:17.607 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T06:25:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:19.260 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:19.304 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:19.706 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-08-20T06:25:19.706 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-08-20T06:25:19.856 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:19.899 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:19.982 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:19.982 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T06:25:19.983 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T06:25:19.983 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:20.025 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:25.888 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-08-20T06:25:25.966 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T06:25:25.991 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T06:25:26.067 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T06:25:26.412 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T06:25:26.525 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:26.631 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T06:25:26.659 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:26.754 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:26.794 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:25:26.870 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T06:25:26.965 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T06:25:27.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T06:25:27.221 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T06:25:27.248 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:27.353 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:27.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:27.430 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T06:25:27.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:27.481 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T06:25:27.740 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:27.776 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T06:25:27.895 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T06:25:27.923 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T06:25:27.958 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T06:25:28.023 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T06:25:28.068 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T06:25:28.109 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T06:25:28.151 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T06:25:28.185 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T06:25:28.213 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T06:25:28.280 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T06:25:28.334 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T06:25:28.366 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T06:25:28.408 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T06:25:28.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T06:25:28.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T06:25:28.590 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T06:25:28.616 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T06:25:28.681 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T06:25:28.731 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T06:25:28.807 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T06:25:28.956 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T06:25:29.009 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T06:25:29.136 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:29.136 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-08-20T06:25:29.136 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-08-20T06:25:29.137 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:29.307 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:29.324 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:29.324 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:29.324 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T06:25:29.324 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:29.536 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T06:25:29.629 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T06:25:29.658 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T06:25:29.702 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T06:25:29.739 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T06:25:29.773 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T06:25:29.857 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T06:25:29.921 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T06:25:29.994 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T06:25:30.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T06:25:30.060 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T06:25:30.107 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T06:25:30.143 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T06:25:30.178 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T06:25:30.220 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T06:25:30.357 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T06:25:30.405 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T06:25:30.447 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T06:25:30.550 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T06:25:30.581 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T06:25:30.631 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T06:25:30.654 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T06:25:30.687 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T06:25:30.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T06:25:30.827 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:30.853 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:30.854 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:30.854 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T06:25:30.854 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:30.854 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:30.854 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:30.914 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T06:25:30.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T06:25:30.984 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T06:25:31.016 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T06:25:31.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T06:25:31.078 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:31.101 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:31.105 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T06:25:31.136 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T06:25:31.158 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T06:25:31.193 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T06:25:31.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T06:25:31.268 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T06:25:31.302 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T06:25:31.389 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T06:25:31.433 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T06:25:31.464 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T06:25:31.553 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:31.559 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T06:25:31.559 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-08-20T06:25:31.560 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:31.576 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:31.577 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:31.577 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T06:25:31.577 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:31.577 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:31.577 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:31.615 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T06:25:31.675 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T06:25:31.731 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T06:25:31.734 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:31.750 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:31.750 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T06:25:31.750 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:31.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T06:25:31.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T06:25:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T06:25:32.098 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T06:25:32.186 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T06:25:32.325 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T06:25:32.347 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T06:25:32.378 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T06:25:32.429 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T06:25:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T06:25:32.463 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T06:25:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T06:25:32.562 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T06:25:32.618 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T06:25:32.663 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T06:25:32.751 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T06:25:32.782 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T06:25:32.868 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:32.936 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T06:25:32.958 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T06:25:32.988 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T06:25:33.001 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T06:25:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:33.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T06:25:33.271 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:33.296 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:33.296 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T06:25:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:33.297 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:33.297 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:33.521 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:33.544 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:33.730 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T06:25:33.837 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T06:25:33.997 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:34.019 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:34.020 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:34.020 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T06:25:34.020 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:34.020 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:34.020 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:34.127 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T06:25:34.183 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T06:25:34.239 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T06:25:34.446 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T06:25:34.497 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T06:25:34.575 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T06:25:34.663 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T06:25:34.793 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T06:25:34.837 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T06:25:34.871 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T06:25:34.899 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T06:25:35.131 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T06:25:35.208 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:25:35.240 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T06:25:35.269 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T06:25:35.319 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:35.347 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:35.379 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T06:25:35.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T06:25:35.478 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:35.600 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T06:25:35.641 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:35.735 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T06:25:35.771 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T06:25:35.807 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T06:25:36.225 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:36.283 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:36.298 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T06:25:36.327 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:36.383 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T06:25:36.404 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T06:25:37.242 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:37.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:37.379 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:37.545 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T06:25:37.568 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:37.592 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:37.593 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:37.593 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T06:25:37.593 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:37.593 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:37.593 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:37.645 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:37.656 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:37.754 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T06:25:37.813 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T06:25:38.640 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:38.701 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T06:25:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:25:38.809 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:25:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:38.844 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:38.864 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:25:38.886 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:25:38.886 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:38.887 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T06:25:38.887 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:38.932 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:25:38.954 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:25:38.955 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:38.955 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T06:25:38.955 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:25:38.955 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:25:38.955 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:39.265 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:25:39.289 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:39.403 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T06:25:39.439 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:39.527 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T06:25:39.528 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T06:25:39.528 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T06:25:39.528 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:39.573 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:39.660 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:39.692 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:39.804 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:39.971 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T06:25:40.003 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:40.026 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:40.026 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:40.027 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T06:25:40.027 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:40.027 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:40.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:40.070 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:40.082 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:40.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T06:25:40.244 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T06:25:41.208 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T06:25:41.239 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:25:41.262 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:25:41.262 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:41.263 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T06:25:41.263 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:25:41.263 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:25:41.263 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:41.306 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:25:41.329 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:25:41.329 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:41.329 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T06:25:41.330 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:41.391 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:25:41.413 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:25:41.413 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:41.413 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T06:25:41.413 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:25:41.414 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:25:41.414 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:41.733 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:25:41.756 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:25:41.756 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:41.756 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T06:25:41.756 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:25:41.757 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:25:41.757 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:43.994 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T06:25:44.043 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T06:25:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T06:25:44.126 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T06:25:44.152 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T06:25:44.358 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T06:25:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T06:25:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T06:25:45.052 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:45.090 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:45.110 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:45.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T06:25:46.028 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T06:25:46.080 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T06:25:46.175 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T06:25:46.207 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T06:25:46.207 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:25:46.291 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:25:46.291 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:46.461 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T06:25:46.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T06:25:46.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T06:25:46.603 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T06:25:46.636 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T06:25:46.838 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T06:25:46.887 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T06:25:46.917 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T06:25:47.538 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:47.581 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:47.601 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:25:48.427 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T06:25:48.511 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T06:25:48.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T06:25:48.665 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T06:25:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T06:25:48.707 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:25:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:25:48.800 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:51.085 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T06:25:51.123 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-08-20T06:25:51.123 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-08-20T06:25:51.123 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-08-20T06:25:51.124 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-08-20T06:25:51.124 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-08-20T06:25:51.124 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-08-20T06:25:51.124 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-08-20T06:25:51.124 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:51.266 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:51.276 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T06:25:51.276 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:51.276 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T06:25:51.276 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:52.517 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:52.517 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:52.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T06:25:52.519 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T06:25:52.520 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T06:25:52.520 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T06:25:52.520 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T06:25:52.520 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T06:25:52.520 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T06:25:52.531 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T06:25:52.532 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T06:25:52.533 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T06:25:52.534 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T06:25:52.535 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T06:25:52.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T06:25:52.537 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T06:25:52.538 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T06:25:52.539 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T06:25:52.540 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T06:25:52.540 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T06:25:52.540 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T06:25:52.540 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T06:25:52.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T06:25:52.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T06:25:52.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T06:25:52.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T06:25:52.541 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T06:25:52.796 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:52.822 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T06:25:52.822 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:52.822 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T06:25:52.822 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:52.823 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T06:25:52.823 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:53.047 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T06:25:53.070 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:53.522 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T06:25:53.545 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:53.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T06:25:53.701 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T06:25:53.756 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T06:25:53.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T06:25:54.014 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T06:25:54.092 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T06:25:54.181 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T06:25:54.320 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T06:25:54.365 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T06:25:54.396 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T06:25:54.424 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T06:25:54.546 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:54.546 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:54.546 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T06:25:54.547 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.548 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.549 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T06:25:54.550 INFO:teuthology.orchestra.run.smithi183.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T06:25:54.551 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T06:25:54.552 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T06:25:54.553 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T06:25:54.554 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T06:25:54.555 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T06:25:54.556 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T06:25:54.557 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T06:25:54.558 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T06:25:54.559 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T06:25:54.560 INFO:teuthology.orchestra.run.smithi183.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T06:25:54.560 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:25:54.560 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-08-20T06:25:54.844 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:54.872 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T06:25:54.877 DEBUG:teuthology.parallel:result is None 2024-08-20T06:25:54.896 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T06:25:54.953 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T06:25:54.979 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:55.315 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:55.315 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T06:25:55.315 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T06:25:55.316 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T06:25:55.317 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T06:25:55.318 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T06:25:55.319 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T06:25:55.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T06:25:55.321 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T06:25:55.322 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T06:25:55.323 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T06:25:55.324 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T06:25:55.325 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T06:25:55.326 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T06:25:55.327 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T06:25:57.320 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:25:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:57.321 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-08-20T06:25:57.321 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.322 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.323 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T06:25:57.324 INFO:teuthology.orchestra.run.smithi107.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T06:25:57.325 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T06:25:57.326 INFO:teuthology.orchestra.run.smithi107.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T06:25:57.327 INFO:teuthology.orchestra.run.smithi107.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T06:25:57.328 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T06:25:57.329 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T06:25:57.330 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T06:25:57.331 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T06:25:57.332 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T06:25:57.333 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T06:25:57.334 INFO:teuthology.orchestra.run.smithi107.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T06:25:57.334 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:25:57.334 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-08-20T06:25:57.667 DEBUG:teuthology.parallel:result is None 2024-08-20T06:25:58.124 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T06:25:58.185 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:59.146 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T06:25:59.176 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:59.288 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T06:25:59.447 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T06:25:59.470 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T06:25:59.494 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:25:59.538 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:59.549 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T06:25:59.647 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T06:25:59.698 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T06:26:00.654 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T06:26:00.690 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:26:00.713 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T06:26:00.714 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:26:00.714 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T06:26:00.714 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:26:00.714 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T06:26:00.714 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:00.758 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:26:00.780 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T06:26:00.780 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:26:00.780 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T06:26:00.780 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:00.825 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T06:26:00.848 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:01.167 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:26:01.190 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T06:26:01.190 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T06:26:01.190 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T06:26:01.190 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:26:01.191 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T06:26:01.191 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:05.920 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T06:26:05.962 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T06:26:06.026 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T06:26:06.051 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T06:26:06.087 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T06:26:06.275 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T06:26:06.316 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T06:26:06.347 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T06:26:06.902 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:26:06.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:26:06.967 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T06:26:07.783 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T06:26:07.860 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T06:26:07.910 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T06:26:07.996 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T06:26:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T06:26:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:26:08.098 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T06:26:08.098 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:26:14.562 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:26:14.562 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T06:26:14.563 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T06:26:14.564 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T06:26:14.564 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T06:26:14.564 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T06:26:14.564 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T06:26:14.564 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T06:26:14.565 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T06:26:14.566 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T06:26:14.567 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T06:26:14.568 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T06:26:14.568 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T06:26:14.568 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T06:26:14.568 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T06:26:14.568 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T06:26:14.569 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T06:26:14.570 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T06:26:14.570 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T06:26:14.570 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T06:26:14.570 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T06:26:14.570 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T06:26:14.571 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T06:26:14.572 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T06:26:14.573 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T06:26:14.574 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T06:26:14.574 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T06:26:14.574 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T06:26:14.574 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T06:26:14.574 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T06:26:14.575 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T06:26:14.576 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T06:26:14.576 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T06:26:14.576 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T06:26:14.576 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T06:26:14.576 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T06:26:14.577 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T06:26:14.578 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T06:26:14.578 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T06:26:14.578 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T06:26:14.578 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T06:26:14.578 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T06:26:14.579 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T06:26:14.580 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T06:26:14.580 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T06:26:14.580 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T06:26:14.580 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T06:26:14.580 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T06:26:14.581 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T06:26:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T06:26:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T06:26:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T06:26:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T06:26:14.582 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T06:26:14.583 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T06:26:14.584 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T06:26:14.585 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T06:26:14.585 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T06:26:14.585 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T06:26:14.585 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T06:26:14.585 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T06:26:14.586 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T06:26:14.587 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-08-20T06:26:16.650 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.651 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.652 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T06:26:16.653 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.654 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T06:26:16.655 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T06:26:16.656 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T06:26:16.657 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T06:26:16.658 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T06:26:16.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T06:26:16.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T06:26:16.661 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T06:26:16.662 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T06:26:16.663 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:26:16.663 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-08-20T06:26:17.024 DEBUG:teuthology.parallel:result is None 2024-08-20T06:26:17.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:17.173 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T06:26:17.199 INFO:teuthology.orchestra.run.smithi107.stdout:19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.200 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.200 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T06:26:17.202 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:17.346 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T06:26:17.372 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.373 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.373 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T06:26:17.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:17.517 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T06:26:17.539 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.539 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T06:26:17.539 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T06:26:17.541 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-20T06:26:17.541 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:26:17.541 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T06:26:17.574 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:26:17.574 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T06:26:17.607 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:26:17.607 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T06:26:17.634 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-20T06:26:17.635 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:26:17.635 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T06:26:17.662 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T06:26:17.727 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:26:17.727 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T06:26:17.757 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T06:26:17.824 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:26:17.824 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T06:26:17.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T06:26:17.913 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-20T06:26:17.913 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:26:17.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T06:26:17.938 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T06:26:18.003 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:26:18.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T06:26:18.031 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T06:26:18.097 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:26:18.098 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T06:26:18.123 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T06:26:18.188 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-20T06:26:18.189 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:26:18.189 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T06:26:18.214 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T06:26:18.279 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:26:18.279 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T06:26:18.303 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T06:26:18.369 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:26:18.369 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T06:26:18.394 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T06:26:18.458 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-20T06:26:18.560 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'mds': {'mds_export_ephemeral_random_max': 0.1}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T06:26:18.560 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:18.561 INFO:tasks.cephadm:Cluster fsid is 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:26:18.561 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-20T06:26:18.561 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.b': '172.21.15.183', 'mon.c': '172.21.15.193'} 2024-08-20T06:26:18.561 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-08-20T06:26:18.561 INFO:tasks.cephadm:First mgr is x 2024-08-20T06:26:18.561 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-20T06:26:18.562 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-08-20T06:26:18.587 DEBUG:teuthology.orchestra.run.smithi183:> sudo hostname $(hostname -s) 2024-08-20T06:26:18.613 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-08-20T06:26:18.639 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-20T06:26:18.639 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:18.782 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82210/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5872-gef427541', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5872.gef427541'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-19 17:15:28.176472', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-08-19-1357', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-08-20T06:26:18.921 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-hemanth-testing-2024-08-19-1357, sha1 ef4275410d8483698016fe484b2a3c664080b4c1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:26:18.923 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T06:26:18.923 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T06:26:18.923 DEBUG:teuthology.orchestra.run.smithi107:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:19.299 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 06:26 /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:19.299 DEBUG:teuthology.orchestra.run.smithi183:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:19.709 INFO:teuthology.orchestra.run.smithi183.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 06:26 /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:19.709 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:20.087 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 06:26 /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:20.087 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:20.107 DEBUG:teuthology.orchestra.run.smithi183:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:20.126 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T06:26:20.161 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 on all hosts... 2024-08-20T06:26:20.162 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T06:26:20.165 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T06:26:20.169 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T06:26:21.790 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T06:26:21.889 INFO:teuthology.orchestra.run.smithi183.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T06:26:21.905 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T06:26:49.553 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-08-20T06:26:49.554 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-08-20T06:26:52.062 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:26:52.063 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:27:02.647 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-08-20T06:27:02.647 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T06:27:02.647 INFO:teuthology.orchestra.run.smithi183.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T06:27:02.647 INFO:teuthology.orchestra.run.smithi183.stdout: "repo_digests": [ 2024-08-20T06:27:02.647 INFO:teuthology.orchestra.run.smithi183.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T06:27:02.648 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-08-20T06:27:02.648 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-08-20T06:27:02.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-08-20T06:27:02.704 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph 2024-08-20T06:27:02.733 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-08-20T06:27:02.767 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-08-20T06:27:02.796 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /etc/ceph 2024-08-20T06:27:02.822 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-08-20T06:27:02.851 INFO:tasks.cephadm:Writing seed config... 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-20T06:27:02.852 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-20T06:27:02.853 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-20T06:27:02.854 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-08-20T06:27:02.855 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:27:02.855 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-20T06:27:02.873 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1c223f02-5ebd-11ef-bccf-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [mds] mds_export_ephemeral_random_max = 0.1 2024-08-20T06:27:02.874 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service 2024-08-20T06:27:02.916 DEBUG:teuthology.orchestra.run.smithi107:mgr.x> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x.service 2024-08-20T06:27:02.959 INFO:tasks.cephadm:Bootstrapping... 2024-08-20T06:27:02.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 -v bootstrap --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:27:03.167 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-08-20T06:27:03.167 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', '-v', 'bootstrap', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-08-20T06:27:03.168 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-20T06:27:03.168 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-08-20T06:27:03.191 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.1 2024-08-20T06:27:03.191 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-08-20T06:27:03.191 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-08-20T06:27:03.199 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T06:27:03.199 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T06:27:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T06:27:03.205 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:27:03.211 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-08-20T06:27:03.217 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-08-20T06:27:03.218 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-08-20T06:27:03.218 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-08-20T06:27:03.240 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.1 2024-08-20T06:27:03.240 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 5.2.1 is present 2024-08-20T06:27:03.240 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-08-20T06:27:03.240 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-08-20T06:27:03.246 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T06:27:03.246 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T06:27:03.252 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T06:27:03.252 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:27:03.259 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-08-20T06:27:03.265 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140296977096560 on /run/cephadm/1c223f02-5ebd-11ef-bccf-c7b262605968.lock 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140296977096560 acquired on /run/cephadm/1c223f02-5ebd-11ef-bccf-c7b262605968.lock 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-08-20T06:27:03.266 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-08-20T06:27:03.267 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-08-20T06:27:03.269 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-08-20T06:27:03.269 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-08-20T06:27:03.272 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-20T06:27:03.272 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-08-20T06:27:03.272 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-08-20T06:27:03.274 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-20T06:27:03.274 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-08-20T06:27:03.275 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-08-20T06:27:03.276 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-20T06:27:03.276 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-20T06:27:03.276 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:0beb2e0cf0f87d34a70e3776822a6ff39fafdc5a4d91bc4481cb39d566d52b0d 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T06:27:06.475 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-20T06:27:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T06:27:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T06:27:07.393 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-08-20T06:27:08.207 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-08-20T06:27:08.207 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-08-20T06:27:09.104 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA8N8Rm8xIpHxAAfhJYSDeKXpEKaR0uwlKafA== 2024-08-20T06:27:09.894 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA9N8Rm/nilFRAAoU31HTeFn0pam620AMzX9w== 2024-08-20T06:27:10.699 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQA+N8RmXxKFChAAJiQ0a1ehWEq2wl8tg0DvWQ== 2024-08-20T06:27:10.701 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = quincy 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T06:27:13.881 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:27:13.882 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-08-20T06:27:15.596 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-08-20T06:27:15.810 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-20T06:27:15.948 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-20T06:27:16.207 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target. 2024-08-20T06:27:16.207 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target. 2024-08-20T06:27:16.436 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a 2024-08-20T06:27:16.436 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-20T06:27:16.606 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target.wants/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@.service. 2024-08-20T06:27:17.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:17 smithi107 ceph-mon[18123]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T06:27:17.129 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T06:27:17.129 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-08-20T06:27:17.137 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-08-20T06:27:17.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:17 smithi107 bash[18102]: 5686c4e2354d4176bc8df644b858e673662631aed4d4029be4ec0e8f146f2f67 2024-08-20T06:27:17.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:17 smithi107 systemd[1]: Started Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:27:18.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-08-20T06:27:18.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:18.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-20T06:27:18.137 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.521378s) 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-08-20T06:27:18.138 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-08-20T06:27:18.250 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T06:27:18.250 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: monmap epoch 1 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: last_changed 2024-08-20T06:27:11.020862+0000 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: min_mon_release 19 (squid) 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: election_strategy: 1 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: fsmap 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: mgrmap e1: no daemons active 2024-08-20T06:27:18.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:18 smithi107 ceph-mon[18123]: from='client.? 172.21.15.107:0/3373228948' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-20T06:27:19.126 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:19.126 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T06:27:19.127 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T06:27:19.128 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-08-20T06:27:19.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:19 smithi107 ceph-mon[18123]: from='client.? 172.21.15.107:0/925415413' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T06:27:19.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:19 smithi107 ceph-mon[18123]: from='client.? 172.21.15.107:0/925415413' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-20T06:27:20.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:20 smithi107 ceph-mon[18123]: from='client.? 172.21.15.107:0/2469148532' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:27:20.400 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-08-20T06:27:20.676 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:20 smithi107 systemd[1]: Stopping Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:27:21.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a[18119]: 2024-08-20T06:27:20.675+0000 7f84122f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:27:21.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a[18119]: 2024-08-20T06:27:20.675+0000 7f84122f1640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-20T06:27:21.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:20 smithi107 podman[18369]: 2024-08-20 06:27:20.77798294 +0000 UTC m=+0.197194985 container died 5686c4e2354d4176bc8df644b858e673662631aed4d4029be4ec0e8f146f2f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-08-20T06:27:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18369]: 2024-08-20 06:27:21.166554978 +0000 UTC m=+0.585766977 container cleanup 5686c4e2354d4176bc8df644b858e673662631aed4d4029be4ec0e8f146f2f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:27:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 bash[18369]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a 2024-08-20T06:27:21.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18381]: 2024-08-20 06:27:21.384151359 +0000 UTC m=+0.596576755 container remove 5686c4e2354d4176bc8df644b858e673662631aed4d4029be4ec0e8f146f2f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-08-20T06:27:21.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-20T06:27:21.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 systemd[1]: Stopped Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:27:21.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 systemd[1]: Starting Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:27:21.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18476]: 2024-08-20 06:27:21.701298711 +0000 UTC m=+0.105561807 container create b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:27:21.962 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18476]: 2024-08-20 06:27:21.611301673 +0000 UTC m=+0.015564770 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18476]: 2024-08-20 06:27:21.887195784 +0000 UTC m=+0.291458883 container init b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 podman[18476]: 2024-08-20 06:27:21.890322743 +0000 UTC m=+0.294585839 container start b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: pidfile_write: ignore empty --pid-file 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: load: jerasure load: lrc 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: RocksDB version: 7.9.2 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Git sha 0 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: DB SUMMARY 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: DB Session ID: BZXCU8P0KC3C325L9OH1 2024-08-20T06:27:21.963 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: CURRENT file: CURRENT 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94827 ; 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.error_if_exists: 0 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.create_if_missing: 0 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.paranoid_checks: 1 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T06:27:21.964 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.env: 0x563dc41f9be0 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.info_log: 0x563dc69ed700 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.statistics: (nil) 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.use_fsync: 0 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_log_file_size: 0 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_fallocate: 1 2024-08-20T06:27:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.use_direct_reads: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.db_log_dir: 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.wal_dir: 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T06:27:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.write_buffer_manager: 0x563dc69f0e60 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T06:27:21.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T06:27:21.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.unordered_write: 0 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.row_cache: None 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.wal_filter: None 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T06:27:21.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.two_write_queues: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.wal_compression: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.atomic_flush: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.log_readahead_size: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T06:27:21.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_background_jobs: 2 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_background_compactions: -1 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_subcompactions: 1 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_open_files: -1 2024-08-20T06:27:21.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_background_flushes: -1 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Compression algorithms supported: 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kZSTD supported: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kXpressCompression supported: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kBZip2Compression supported: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kLZ4Compression supported: 1 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kZlibCompression supported: 1 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: kSnappyCompression supported: 1 2024-08-20T06:27:21.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.merge_operator: 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_filter: None 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x563dc69ed380) 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-08-20T06:27:21.973 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 4 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x563dc69e1610 2024-08-20T06:27:21.974 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-08-20T06:27:21.975 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: bloomfilter 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 5 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: max_auto_readahead_size: 262144 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: prepopulate_block_cache: 0 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: initial_auto_readahead_size: 8192 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression: NoCompression 2024-08-20T06:27:21.976 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.num_levels: 7 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:27:21.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:27:21.978 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T06:27:21.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T06:27:21.980 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.inplace_update_support: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.bloom_locality: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.max_successive_merges: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T06:27:21.981 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.ttl: 2592000 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enable_blob_files: false 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.min_blob_size: 0 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T06:27:21.982 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6c061cee-c7a3-4b48-bf69-d6816829e89f 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135241916074, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135241916963, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90981, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 282, "table_properties": {"data_size": 89029, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11595, "raw_average_key_size": 48, "raw_value_size": 82524, "raw_average_value_size": 342, "num_data_blocks": 12, "num_entries": 241, "num_filter_entries": 241, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724135241, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6c061cee-c7a3-4b48-bf69-d6816829e89f", "db_session_id": "BZXCU8P0KC3C325L9OH1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135241917066, "job": 1, "event": "recovery_finished"} 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-20T06:27:21.983 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x563dc6a10e00 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: DB pointer 0x563dc6b0e000 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-08-20T06:27:21.984 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 90.71 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 90.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 150.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 150.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.985 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 28.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 28.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache BinnedLRUCache@0x563dc69e1610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.986 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???) e1 preinit fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).mds e1 new map 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).mds e1 print_map 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: btime 2024-08-20T06:27:17:087657+0000 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-08-20T06:27:21.987 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 bash[18476]: b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d 2024-08-20T06:27:21.988 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 systemd[1]: Started Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:27:21.988 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: monmap epoch 1 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: last_changed 2024-08-20T06:27:11.020862+0000 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: min_mon_release 19 (squid) 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: election_strategy: 1 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: fsmap 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T06:27:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:21 smithi107 ceph-mon[18495]: mgrmap e1: no daemons active 2024-08-20T06:27:22.987 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-20T06:27:23.009 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:27:23.009 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-08-20T06:27:23.009 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-20T06:27:23.010 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-20T06:27:23.245 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x 2024-08-20T06:27:23.245 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-20T06:27:23.391 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968.target.wants/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@.service. 2024-08-20T06:27:23.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:23 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/676767348' entity='client.admin' 2024-08-20T06:27:23.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:23 smithi107 podman[18709]: 2024-08-20 06:27:23.808351747 +0000 UTC m=+0.301960097 container start b70a6c5efcff28242c8a573086e3a87bddb6095c3aa1c5ff7a0344b4315a3eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-20T06:27:23.904 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T06:27:23.904 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-08-20T06:27:23.912 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T06:27:23.912 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:27:23.912 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-08-20T06:27:23.912 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T06:27:23.921 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T06:27:23.921 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-08-20T06:27:23.929 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-08-20T06:27:24.141 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:23 smithi107 bash[18709]: b70a6c5efcff28242c8a573086e3a87bddb6095c3aa1c5ff7a0344b4315a3eb5 2024-08-20T06:27:24.141 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:23 smithi107 systemd[1]: Started Ceph mgr.x for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:27:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:24 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4288112917' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T06:27:24.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:24.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:24.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:27:24.966 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T06:27:24.967 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:24.968 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T06:27:24.969 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T06:27:24.970 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:24.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T06:27:17:087657+0000", 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T06:27:17.088359+0000", 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:24.973 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-08-20T06:27:27.783 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:27 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2237056600' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T06:27:28.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T06:27:28.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T06:27:28.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T06:27:28.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T06:27:28.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T06:27:28.069 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T06:27:17:087657+0000", 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T06:27:28.070 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:28.071 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T06:27:17.088359+0000", 2024-08-20T06:27:28.072 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:28.072 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:28.072 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T06:27:28.072 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:28.072 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (2/15)... 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: Activating manager daemon x 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: mgrmap e2: x(active, starting, since 0.00307928s) 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: Manager daemon x is now available 2024-08-20T06:27:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T06:27:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T06:27:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' 2024-08-20T06:27:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' 2024-08-20T06:27:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:28 smithi107 ceph-mon[18495]: from='mgr.14100 172.21.15.107:0/1467722074' entity='mgr.x' 2024-08-20T06:27:31.022 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:30 smithi107 ceph-mon[18495]: mgrmap e3: x(active, since 1.00683s) 2024-08-20T06:27:31.023 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/823365825' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T06:27:31.322 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:31.323 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T06:27:31.324 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T06:27:31.325 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T06:27:17:087657+0000", 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T06:27:31.326 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T06:27:17.088359+0000", 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:31.327 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-08-20T06:27:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:31 smithi107 ceph-mon[18495]: mgrmap e4: x(active, since 2s) 2024-08-20T06:27:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:31 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2389839045' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T06:27:32.433 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-08-20T06:27:32.434 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T06:27:32.435 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-08-20T06:27:33.435 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:33 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/867459437' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-20T06:27:33.435 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ignoring --setuser ceph since I am not root 2024-08-20T06:27:33.435 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ignoring --setgroup ceph since I am not root 2024-08-20T06:27:34.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:34 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/867459437' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-20T06:27:34.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:34 smithi107 ceph-mon[18495]: mgrmap e5: x(active, since 4s) 2024-08-20T06:27:34.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:34.827 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-08-20T06:27:34.828 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 5... 2024-08-20T06:27:35.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:35 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3823479004' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: Active manager daemon x restarted 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: Activating manager daemon x 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: osdmap e2: 0 total, 0 up, 0 in 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: mgrmap e6: x(active, starting, since 0.00328741s) 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T06:27:38.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T06:27:38.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:38 smithi107 ceph-mon[18495]: Manager daemon x is now available 2024-08-20T06:27:39.739 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-20T06:27:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T06:27:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 5 is available 2024-08-20T06:27:39.740 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-08-20T06:27:40.340 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: mgrmap e7: x(active, since 1.00496s) 2024-08-20T06:27:40.340 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:40.340 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:40.340 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: Found migration_current of "None". Setting to last migration. 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T06:27:40.341 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:40 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T06:27:41.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-mon[18495]: mgrmap e8: x(active, since 2s) 2024-08-20T06:27:41.270 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:41.271 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:27:41.611 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:27:41.434+0000 7f362805a640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:06:27:41] ENGINE Error in HTTPServer.serve 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: Traceback (most recent call last): 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._connections.run(self.expiration_interval) 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._run(expiration_interval) 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: s = self.context.wrap_socket( 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: return self.sslsocket_class._create( 2024-08-20T06:27:41.612 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T06:27:41.613 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self.do_handshake() 2024-08-20T06:27:41.613 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T06:27:41.613 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._sslobj.do_handshake() 2024-08-20T06:27:41.613 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T06:27:41.613 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:27:41.935 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-20T06:27:41.935 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-08-20T06:27:42.326 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-mon[18495]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:42.327 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:27:42.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: Generating public/private rsa key pair. 2024-08-20T06:27:42.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: Your identification has been saved in /tmp/tmpqa1tjw3i/key 2024-08-20T06:27:42.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: Your public key has been saved in /tmp/tmpqa1tjw3i/key.pub 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: The key fingerprint is: 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: SHA256:9jzGM/jyk1QYVHbnkJk7H0u0ox1sqD4eE4yv+6NPIFY ceph-1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: The key's randomart image is: 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: +---[RSA 3072]----+ 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | ...o o+.| 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | .. .+= | 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | E o +.o| 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | . + ..oO | 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | oSo +. =o=| 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | ...=+... o.| 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | ..X* | 2024-08-20T06:27:42.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | .o=Bo | 2024-08-20T06:27:42.863 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: | *O=o | 2024-08-20T06:27:42.863 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: +----[SHA256]-----+ 2024-08-20T06:27:43.180 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: [20/Aug/2024:06:27:41] ENGINE Bus STARTING 2024-08-20T06:27:43.181 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: [20/Aug/2024:06:27:41] ENGINE Serving on https://172.21.15.107:7150 2024-08-20T06:27:43.181 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: [20/Aug/2024:06:27:41] ENGINE Error in HTTPServer.serve 2024-08-20T06:27:43.181 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-08-20T06:27:43.181 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T06:27:43.181 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-08-20T06:27:43.182 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: [20/Aug/2024:06:27:41] ENGINE Serving on http://172.21.15.107:8765 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: [20/Aug/2024:06:27:41] ENGINE Bus STARTED 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:43.183 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:43 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:44.389 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:27:44.389 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-20T06:27:44.389 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-08-20T06:27:44.394 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-08-20T06:27:44.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:44 smithi107 ceph-mon[18495]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:44.511 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:44 smithi107 ceph-mon[18495]: Generating ssh key... 2024-08-20T06:27:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:45 smithi107 ceph-mon[18495]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:46.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:46 smithi107 ceph-mon[18495]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:47 smithi107 ceph-mon[18495]: Deploying cephadm binary to smithi107 2024-08-20T06:27:48.736 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-08-20T06:27:48.737 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-08-20T06:27:49.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:49 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:49.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:49 smithi107 ceph-mon[18495]: Added host smithi107 2024-08-20T06:27:49.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:49 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:27:49.951 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-20T06:27:49.951 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-08-20T06:27:50.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:50 smithi107 ceph-mon[18495]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:50.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:50 smithi107 ceph-mon[18495]: Saving service mon spec with placement count:5 2024-08-20T06:27:50.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:50 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:51.123 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-20T06:27:51.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:51 smithi107 ceph-mon[18495]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:27:51.698 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:51 smithi107 ceph-mon[18495]: Saving service mgr spec with placement count:2 2024-08-20T06:27:51.699 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:51 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:52.747 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2407927894' entity='client.admin' 2024-08-20T06:27:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:54 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/859723222' entity='client.admin' 2024-08-20T06:27:54.567 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-08-20T06:27:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:55 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:55 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:56.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:56 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3723421342' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-20T06:27:56.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:56 smithi107 ceph-mon[18495]: from='mgr.14118 172.21.15.107:0/1946570657' entity='mgr.x' 2024-08-20T06:27:56.483 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ignoring --setuser ceph since I am not root 2024-08-20T06:27:56.483 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:27:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ignoring --setgroup ceph since I am not root 2024-08-20T06:27:57.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3723421342' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-20T06:27:57.490 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:57 smithi107 ceph-mon[18495]: mgrmap e9: x(active, since 18s) 2024-08-20T06:27:58.075 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:27:58.075 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-20T06:27:58.075 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T06:27:58.075 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T06:27:58.075 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T06:27:58.076 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:27:58.076 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-08-20T06:27:58.076 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-08-20T06:27:58.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:27:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2574461214' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: Active manager daemon x restarted 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: Activating manager daemon x 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: osdmap e3: 0 total, 0 up, 0 in 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: mgrmap e10: x(active, starting, since 0.00416401s) 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T06:28:01.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:01 smithi107 ceph-mon[18495]: Manager daemon x is now available 2024-08-20T06:28:02.586 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: [20/Aug/2024:06:28:02] ENGINE Bus STARTING 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: [20/Aug/2024:06:28:02] ENGINE Serving on https://172.21.15.107:7150 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: [20/Aug/2024:06:28:02] ENGINE Error in HTTPServer.serve 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T06:28:02.587 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T06:28:02.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-mon[18495]: mgrmap e11: x(active, since 1.00575s) 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:28:02.134+0000 7f849208b640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:06:28:02] ENGINE Error in HTTPServer.serve 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: Traceback (most recent call last): 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._connections.run(self.expiration_interval) 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._run(expiration_interval) 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T06:28:02.589 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: s = self.context.wrap_socket( 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: return self.sslsocket_class._create( 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self.do_handshake() 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: self._sslobj.do_handshake() 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T06:28:02.590 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:28:02 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:28:03.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-08-20T06:28:03.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-20T06:28:03.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T06:28:03.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-08-20T06:28:03.285 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-08-20T06:28:03.285 INFO:teuthology.orchestra.run.smithi107.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-20T06:28:04.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-20T06:28:04.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-20T06:28:04.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: [20/Aug/2024:06:28:02] ENGINE Serving on http://172.21.15.107:8765 2024-08-20T06:28:04.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: [20/Aug/2024:06:28:02] ENGINE Bus STARTED 2024-08-20T06:28:04.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:04.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:05.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:04 smithi107 ceph-mon[18495]: mgrmap e12: x(active, since 2s) 2024-08-20T06:28:05.035 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:04 smithi107 ceph-mon[18495]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:05.346 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE/DCCAuSgAwIBAgIUVCrucan7mCOxu80DEvIkZ9fURMEwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgyMDA2MjgwNFoXDTM0\nMDgyMTA2MjgwNFowGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjEwNzCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBANHBfB3o7yC3pkkbIN8YYbT766f2GudRA8Yb\n2PBAkWgNLiTt6N9YQqrzbMb56dwIvRkjyZ70iyB+YUaPsNK17ULm1GTe1zfVwtkb\noYhR0iNmAMMMoqyZhoG3w1PjQiYBbq7xAYWT5s4YD5zOpgkts7KEzqiyIJ6bbmzy\nsAP131InogjCJt7irFDCGNIZjtQM45QWtxhGlxKgV3cc0klxaGxfLaJ0ln/ImsNB\ncWXN1QD5SPE48Os0eXZB2EZSiRb99XVg30gtbRxGjQrLGqqo2quqVj294vEKpNtL\n1sSEq6ENx79XcdP8G+VKIEhwTO75Zkd4iAzyquJ/xVgIjxQfcqMVOtIJhVo321JF\nyz4sA+AGAdcvp6etqCuMDlzB7Vj2MM2B+7xSAbFoWNaLYUR9lWlUSBEH9vmpEWLB\nYyGMJjOvXmImMFyYvsE4wS8iKYDpICp2afvtv+FU3+AR6kdqSYu6S163sZ9m+BLv\nxYgxyZrB5YHdTPohA4+BzH1rGZ2ewZSL10WW7sIe2FiCOtNoiEzpqLCSUpsXOAcb\ndXV1My14yKJnGWeMk8JvmaTClf9vvUQI3D81ZOJzu1V3IH5fQdpZCrjua4ekxitQ\nIYyVY4R0NktXDgrtuUOrFEwzN8t7LqLF/+3BE00kL05OxceW+MWgMZHs7sSeFUhV\n1NQJMe8bAgMBAAGjPzA9MC0GA1UdEQQmMCSCCXNtaXRoaTEwN4IRZGFzaGJvYXJk\nX3NlcnZlcnOHBKwVD2swDAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAgEA\n0z+NIxdDW5PGYNNMaM6ihkw6EF32mQNTl0a0AlPeUyAzpb/uT+bs7yp0nQKSy6Sa\nY8hUL1j/IPL+XKKBGm4gFhtk8uaGq952j15QnSvqEhJdfxKSEJBFz67Hui4LhKz+\n8pKTz02Yf/xnkWLUiBIyvBGGS5MOR8ew2v5c60uUDnmLrRDr8PoP6SRK1W+B52F5\nqXyGAfOKb2XMaNhHXfZ6R6Hn8Pa735GvGAyWirD/KkpA9xOdfX7OrnCv6M/Ds59V\nqvNzAp4hWNh6WnI4yLDHrzfw7xhAG/Y5ZUIdPlAhPVpx5APNor/jpJKkMhDENT+T\no9kts7qXi/91OP2mNr66S2RkVeXYbIR7C4uW4CrAeWc98HuFdYGmnsGkE599BRkt\nvVQcetDlkdPd1DDRlOvj/lcQCMYmlwlpHi2s4VkxCm/mI/XI2LDl/oVXzGxBPigi\nC0lSC+8p86llScneOWiRoK8zvbHxN7DejKGpxEW+cb0krOQQoAXYLdYpsPy1ofry\ndBu+w2Vch+UJf5W0THA8zGMclyzp4qLEqgQqDOFumNCL3RSGDzR5jMGGnKiDNpoW\neovC+UoXW3URrByuEiYNc9eGPrHIBash7GBXwhbPlwvVelux4anzYxvbL3ZHK70z\nt1nWnvB/VN5M4kzicaCVunZsEZwEOUxjxmiyh03tEiE=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEA0cF8HejvILemSRsg3xhhtPvrp/Ya51EDxhvY8ECRaA0uJO3o\n31hCqvNsxvnp3Ai9GSPJnvSLIH5hRo+w0rXtQubUZN7XN9XC2RuhiFHSI2YAwwyi\nrJmGgbfDU+NCJgFurvEBhZPmzhgPnM6mCS2zsoTOqLIgnptubPKwA/XfUieiCMIm\n3uKsUMIY0hmO1AzjlBa3GEaXEqBXdxzSSXFobF8tonSWf8iaw0FxZc3VAPlI8Tjw\n6zR5dkHYRlKJFv31dWDfSC1tHEaNCssaqqjaq6pWPb3i8Qqk20vWxISroQ3Hv1dx\n0/wb5UogSHBM7vlmR3iIDPKq4n/FWAiPFB9yoxU60gmFWjfbUkXLPiwD4AYB1y+n\np62oK4wOXMHtWPYwzYH7vFIBsWhY1othRH2VaVRIEQf2+akRYsFjIYwmM69eYiYw\nXJi+wTjBLyIpgOkgKnZp++2/4VTf4BHqR2pJi7pLXrexn2b4Eu/FiDHJmsHlgd1M\n+iEDj4HMfWsZnZ7BlIvXRZbuwh7YWII602iITOmosJJSmxc4Bxt1dXUzLXjIomcZ\nZ4yTwm+ZpMKV/2+9RAjcPzVk4nO7VXcgfl9B2lkKuO5rh6TGK1AhjJVjhHQ2S1cO\nCu25Q6sUTDM3y3suosX/7cETTSQvTk7Fx5b4xaAxkezuxJ4VSFXU1Akx7xsCAwEA\nAQKCAgATZ7IC1lzhCvxwQPk+inPnmXLIvCTrB3yw5DyipWaOWMHPGs9zcifgxtpm\nYhJ3j7tmyv/JG9iPLgHz1DFInRoRGWfG2TLR8I2xJjRAd0lYiwRBBzc5fo6KC6Uf\naqPg/jruJk+8ut0cbe9r01Za/QZl2pF/29LWuYxyudT82M9LO21rIAMWlaPZaMuB\nDGpBRdv8BEn2DfnF+F1/IIR31O0TMY74O478zWM78x75ixxMlcNgxaLML7OY9Ch9\nAP+7x8zgihC+Da3Nct1zhSy3Yx2vyfdmiN2o7RIpU4/P6HI0WSEEm6NlOCqggMYR\nzYngheQncogronuarrVkGo6aDvq83r3xBauzl+rUjR0kvZ7XYS6sk1hY0eKd90zU\nFkEfNCCTFXf5j768TmMPr6GjvBKw5+5jOOdExfyO2omdlIjoXGZngwosULILsarH\n8QURMbzt7opiT1K7iI0psP/X7NZlWt3HjVAqU1IK2CjGHFKJJW0Xs3yRi3JsrS/U\nWtU2Jk7uzB5ZpjBk1mnUXbcrLLGdmi17ws8FYqi9kFvg6QFVktLW/mfTdyAZgXcH\ntojwQ2/yxGsSnMhFHCJoJ4iPM4MDj0twVINPkQA9IG5Mue0ZAtVSQMv39BcHmBK/\nO9oYzmrWReq3fmg4m1M7RtCIKq1H4o/pV79y7R0AgQoq7pJq2QKCAQEA8SaEDKw0\nimCyzNEEZtiTaMooOPDZNvksVIrv+9Eg1uC3I8k0oGeC5fLnPgmOxw9WWjx9ge22\nmPzylh9HwuLC/qlqejSmEKafGK4Ob4rvx6j3Q2GPA4ILU6VZoxriuRaZPWUtbiDs\nzsDIRMm/+NOEmhPtfsSbCgntVsoM73yt96bKTKBkGNOvOIKswL4V2/7xDUNVnIfL\nzxo3/wHTKmnIL765VwA9N/FVRuPjVBNfIMH5mkzfTXLD1J97UQ+RxHFFJFZTSM27\nR7aIdfyWaOuN3fEcSvoQ+OvhPyarfeDx+6IshMZQXKHk0VkkUEI8PvwohQ/zkyYx\ngnmaVnAUTnRePQKCAQEA3qwQsh/m/fshhdODVUuEApfrrhfe9o9AVCZDj3blRQcT\nB85VNNyuZg0CZmjWR4Ap64IMZYu0FXhuAvbUlhxC6D2QSlWBzNxe8GJwjOniYGpV\n+LWMIWIncQx9z/xSdxYO73IEvOOY5v06RtiOdPm9fPkYYIxMARABsY3LsxIQewUg\nABWTkVuXBk/PqjMbizyKx55RMbdg9zYT+SJiijIglM/nPKbJrvfxt2b1mImEYg6/\nP5J/c/aZjwrIHzIfexiSEX2QlxCEBZUkEcbW1Z9nB9ZCbDYPqBVN2sADfFLlIVI/\nwT8g9HqRw4mWdsYhm5FCHxESLwUi+y8DVEa6t0NwNwKCAQEAp437clK0rvFBSs5w\nIIWa5tSP3gB67Jd1FcjztiQ4jRTWKydCSbGiYePEiHZbnjQp68jb68sfdG1MoMJb\nOi4VDpzoPsEWIbukD6HgpeADetcL9uVyWR2igRwCjDHCFph9IQuL5Vk7MO0COcl0\neY9+MxBllA9HKlJ7djItirk2pzxck6Fd62Ip/TnOjp2E0rbxBiYi9f/kvKGbBrRd\n5LpbPdFnNewUnSox9m+hJGnRTxF4PJ3tCgMTzkiBhnEq9zM8t2LDu3I3Djzl1BUM\n7Rm+GNfOsMnPwSUj/J0ByxThDdZA2IBylepm8o7djGZ5Nco+3TWYKEYBYIbdlfGK\nKnjwgQKCAQAr4zQDN6B6jKXFz7OB+zlQnYrZLGFIrWFbdJM6Uo/LtCI/Fp7apWhG\nOugvMRiSzSL2BRLWbNuYo7zjHq01zOovBftLe2Dy9LhNnslaLLruj+bT90xgRA1M\nDc86UBwc1p8o+IsQg7deR6jVz9b8HUEIwbvZQ62zQ+lhinYMnFHdPVxaR+Zm7uts\nVgyzGhGittMPlxefueoFr4XuQ6WyYUDks7l6zdDQiJP+tu9PGkOaZBezuvcd7wvD\nErfw1hs4UtZvVhVKgPnUc9nBGzS6tAlFlShQVbNuSIyowdGuDlSU5bYlK4EeTo4u\nQQ8D5/c1x0KGBwJTfCqx4UlBshMZDdSvAoIBAE/f5f63/smVLFP7G30pX6od7OwG\n0Lx6dlNxe+m8UBlFB1IYhTzWVJlbAKJRsWwthw8wRbqV9GbmxBXZV79vONDwi4OC\nQRTijlJav+u185xPtL43HfKz08mp0JxhTMXXH1zwCCyNmOn+uemriBCGuMWDW2A8\nQIW3Oz/5OV0chDbvptUh5wq25ct4WPyW/oYv6k5KdGZ7LswfwYsREUCITa+eMEFr\nBh+YA+qFhZ/g10Eta1o6tI+ky5L53790aSCqizEBbhG/vbMvGv0eC5ncUFB1MSnC\npijOrREuqqNPdvjskpjfSIPSRLcU1K8OYdX+OVYPPnn3lZBVJJDdQKIpdm8=\n-----END RSA PRIVATE KEY-----\n"} 2024-08-20T06:28:06.510 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-20T06:28:07.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:06 smithi107 ceph-mon[18495]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:07.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:06 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-20T06:28:07.691 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-08-20T06:28:08.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:08 smithi107 ceph-mon[18495]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:08.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:08 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:09.139 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$CW/T0UTs8oKZtkp0K1x2Ke581jqEcGVx7U6QhAu8ilRspeoJKFEOm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724135288, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-20T06:28:09.139 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-08-20T06:28:09.707 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:09 smithi107 ceph-mon[18495]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:09.707 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:10.277 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-08-20T06:28:10.287 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T06:28:10.287 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-08-20T06:28:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T06:28:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-08-20T06:28:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-08-20T06:28:10.294 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-20T06:28:10.295 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107:8443/ 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout: Password: s91pr2j6io 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:10.296 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config directory 2024-08-20T06:28:10.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1144246885' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.450 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:11.451 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-08-20T06:28:11.491 INFO:tasks.cephadm:Fetching config... 2024-08-20T06:28:11.492 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:11.492 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-20T06:28:11.509 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-20T06:28:11.509 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:11.509 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-20T06:28:11.565 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-20T06:28:11.566 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:11.566 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-20T06:28:11.630 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-20T06:28:11.631 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:11.631 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-20T06:28:11.686 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-20T06:28:11.686 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T06:28:11.760 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:11.772 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T06:28:11.891 INFO:teuthology.orchestra.run.smithi183.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:11.905 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T06:28:11.958 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDx/IVpdFlixCHhzVIOu/WzwZHO2GvkVlHITZ0mIYTRMvqVo8HigW71I7mzLmKZUPj09A762a0gVkJD038qeGGDirvjp6pvWCwUMrXvz8tyq8mpl3L/ciQCFGGA28euWEtcdsjyQLhI2gFF1Pz9nAry8Mq6Borb6jt9WlJbHeoKxxIAjZK7/vsoaO8dwzdIShC4w2Mjib8XIT50R0lUS+qMUIJt4rLPTwP32H72oRtUKRpq8BHlcr8juDAy4N3ubXFF68rxpiM9mgVyVlFakNkuJ0F3kI42gEla06U3YlFOoJI1bTW1sPq++hxkearge2erKZ3eRuouQ4g+CTwf4MKgufXARf5FrDJb1XhbGkoglrvqp+W7x7og8oHKEVjhLvVlYW22hNzsCptmJbIFUPkNrqxkVhIAHdiA6lG9EFVY66Hnl4oVNV1A1QNdludABor92ixW5RgZGFgwi1Rqd8BTkIWGfdjxEO/hHtxre87/t2iAL+ZmW4bnikURlckPyMk= ceph-1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:11.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-20T06:28:12.043 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:11 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1920731209' entity='client.admin' 2024-08-20T06:28:12.226 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:13.599 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-20T06:28:13.599 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-20T06:28:14.181 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:14.295 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:13 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4280733834' entity='client.admin' 2024-08-20T06:28:15.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: mgrmap e13: x(active, since 12s) 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:15.299 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:15.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:15.300 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:15.521 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi183 2024-08-20T06:28:15.521 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:28:15.521 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.conf 2024-08-20T06:28:15.543 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:28:15.543 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:15.601 INFO:tasks.cephadm:Adding host smithi183 to orchestrator... 2024-08-20T06:28:15.601 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch host add smithi183 2024-08-20T06:28:15.852 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: Updating smithi107:/etc/ceph/ceph.conf 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: Updating smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:16.207 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:16 smithi107 ceph-mon[18495]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:16 smithi107 ceph-mon[18495]: Updating smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:18.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:17 smithi107 ceph-mon[18495]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi183", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:18 smithi107 ceph-mon[18495]: Deploying cephadm binary to smithi183 2024-08-20T06:28:19.606 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi183' with addr '172.21.15.183' 2024-08-20T06:28:20.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T06:28:20.518 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:20 smithi107 ceph-mon[18495]: Added host smithi183 2024-08-20T06:28:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:21.159 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:21.159 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}] 2024-08-20T06:28:21.835 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-08-20T06:28:21.835 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:28:21.835 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-08-20T06:28:21.854 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:28:21.854 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:21.912 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-08-20T06:28:21.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch host add smithi193 2024-08-20T06:28:22.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:22.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:21 smithi107 ceph-mon[18495]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:28:22.150 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:22 smithi107 ceph-mon[18495]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:24.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:24.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:24.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: Deploying cephadm binary to smithi193 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:26.027 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-08-20T06:28:26.651 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T06:28:26.885 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:27.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:27.258 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:27.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: Added host smithi193 2024-08-20T06:28:27.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:27.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:27.259 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:27.528 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:28:27.529 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.183", "hostname": "smithi183", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-08-20T06:28:28.140 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-20T06:28:28.141 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-20T06:28:28.387 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:28.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:28 smithi107 ceph-mon[18495]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:28.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:28.620 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:28 smithi107 ceph-mon[18495]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:28:29.372 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-08-20T06:28:29.675 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:29 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T06:28:29.988 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-08-20T06:28:29.988 INFO:tasks.cephadm:Adding mon.b on smithi183 2024-08-20T06:28:29.988 INFO:tasks.cephadm:Adding mon.c on smithi193 2024-08-20T06:28:29.988 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c' 2024-08-20T06:28:30.274 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-20T06:28:30.319 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:30.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:30.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:30.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:30 smithi107 ceph-mon[18495]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:31.006 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-08-20T06:28:31.618 DEBUG:teuthology.orchestra.run.smithi183:mon.b> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.b.service 2024-08-20T06:28:31.621 DEBUG:teuthology.orchestra.run.smithi193:mon.c> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.c.service 2024-08-20T06:28:31.624 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T06:28:31.624 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T06:28:32.288 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:32.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:32.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:32.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:32.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:32.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:32.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:31 smithi107 ceph-mon[18495]: Deploying daemon mon.c on smithi193 2024-08-20T06:28:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:33 smithi107 ceph-mon[18495]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:34.490 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:28:34.490 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","modified":"2024-08-20T06:27:11.020862Z","created":"2024-08-20T06:27:11.020862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-20T06:28:34.491 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-08-20T06:28:35.221 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 podman[21924]: 2024-08-20 06:28:35.195584045 +0000 UTC m=+0.088466117 container create c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:28:35.221 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 podman[21924]: 2024-08-20 06:28:35.123574564 +0000 UTC m=+0.016456650 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:28:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:35 smithi107 ceph-mon[18495]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:35 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/2798940779' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 podman[21924]: 2024-08-20 06:28:35.381104522 +0000 UTC m=+0.273986597 container init c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 podman[21924]: 2024-08-20 06:28:35.384306217 +0000 UTC m=+0.277188292 container start c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pidfile_write: ignore empty --pid-file 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: load: jerasure load: lrc 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: RocksDB version: 7.9.2 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Git sha 0 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: DB SUMMARY 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: DB Session ID: JHXK8CO31OVI3TG4CTTK 2024-08-20T06:28:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: CURRENT file: CURRENT 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.error_if_exists: 0 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.create_if_missing: 0 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.paranoid_checks: 1 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.env: 0x55e452a1dbe0 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T06:28:35.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.info_log: 0x55e4532d7700 2024-08-20T06:28:35.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T06:28:35.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.statistics: (nil) 2024-08-20T06:28:35.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.use_fsync: 0 2024-08-20T06:28:35.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_log_file_size: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_fallocate: 1 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.use_direct_reads: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.db_log_dir: 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.wal_dir: 2024-08-20T06:28:35.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.write_buffer_manager: 0x55e4532dae60 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T06:28:35.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.unordered_write: 0 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.row_cache: None 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.wal_filter: None 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T06:28:35.561 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.two_write_queues: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.wal_compression: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.atomic_flush: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.log_readahead_size: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T06:28:35.562 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_background_jobs: 2 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_background_compactions: -1 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_subcompactions: 1 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T06:28:35.563 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_open_files: -1 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_background_flushes: -1 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Compression algorithms supported: 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kZSTD supported: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kXpressCompression supported: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kBZip2Compression supported: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kLZ4Compression supported: 1 2024-08-20T06:28:35.564 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kZlibCompression supported: 1 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: kSnappyCompression supported: 1 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.merge_operator: 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_filter: None 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T06:28:35.565 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e4532d7380) 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_type: 0 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_index_type: 0 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_shortening: 1 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: checksum: 4 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: no_block_cache: 0 2024-08-20T06:28:35.566 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache: 0x55e4532cb610 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_options: 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: capacity : 536870912 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_shard_bits : 4 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: strict_capacity_limit : 0 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_cache_compressed: (nil) 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: persistent_cache: (nil) 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size: 4096 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_size_deviation: 10 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_restart_interval: 16 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: index_block_restart_interval: 1 2024-08-20T06:28:35.567 INFO:journalctl@ceph.mon.c.smithi193.stdout: metadata_block_size: 4096 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: partition_filters: 0 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: use_delta_encoding: 1 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: filter_policy: bloomfilter 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: whole_key_filtering: 1 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: verify_compression: 0 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: format_version: 5 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_index_compression: 1 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: block_align: 0 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: max_auto_readahead_size: 262144 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: prepopulate_block_cache: 0 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: initial_auto_readahead_size: 8192 2024-08-20T06:28:35.568 INFO:journalctl@ceph.mon.c.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression: NoCompression 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.num_levels: 7 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T06:28:35.569 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:28:35.570 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T06:28:35.571 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T06:28:35.572 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.inplace_update_support: 0 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T06:28:35.573 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.bloom_locality: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.max_successive_merges: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.ttl: 2592000 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enable_blob_files: false 2024-08-20T06:28:35.574 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.min_blob_size: 0 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 41888efc-05c7-4ddf-b3dc-4b25c4ffe50a 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135315411959, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-20T06:28:35.575 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135315412520, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724135315, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "41888efc-05c7-4ddf-b3dc-4b25c4ffe50a", "db_session_id": "JHXK8CO31OVI3TG4CTTK", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135315412647, "job": 1, "event": "recovery_finished"} 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e4532fae00 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: DB pointer 0x55e4533f8000 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** DB Stats ** 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:35.576 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.577 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** Compaction Stats [default] ** 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T06:28:35.578 INFO:journalctl@ceph.mon.c.smithi193.stdout: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache BinnedLRUCache@0x55e4532cb610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: using public_addr v2:172.21.15.193:0/0 -> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: starting mon.c rank -1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(???) e0 preinit fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 bash[21924]: c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).mds e1 new map 2024-08-20T06:28:35.579 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: e1 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: btime 2024-08-20T06:27:17:087657+0000 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: legacy client fscid: -1 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout: No filesystems configured 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T06:28:35.580 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:35.581 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Deploying cephadm binary to smithi193 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:35.582 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Added host smithi193 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T06:28:35.583 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:35.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: Deploying daemon mon.c on smithi193 2024-08-20T06:28:35.585 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.586 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:35.586 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/2798940779' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:35.586 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 ceph-mon[21944]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T06:28:35.586 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:35 smithi193 systemd[1]: Started Ceph mon.c for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:28:36.000 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T06:28:36.001 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T06:28:36.260 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:28:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: mon.a calling monitor election 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: mon.c calling monitor election 2024-08-20T06:28:40.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: monmap epoch 2 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: last_changed 2024-08-20T06:28:35.439375+0000 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: min_mon_release 19 (squid) 2024-08-20T06:28:40.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: election_strategy: 1 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: fsmap 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: mgrmap e13: x(active, since 39s) 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: overall HEALTH_OK 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:40.808 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:40.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:40.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: mon.a calling monitor election 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: mon.c calling monitor election 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: monmap epoch 2 2024-08-20T06:28:40.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: last_changed 2024-08-20T06:28:35.439375+0000 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: min_mon_release 19 (squid) 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: election_strategy: 1 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: fsmap 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: mgrmap e13: x(active, since 39s) 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: overall HEALTH_OK 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:40.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:40.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:41.338 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:28:41.338 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","modified":"2024-08-20T06:28:35.439375Z","created":"2024-08-20T06:27:11.020862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T06:28:41.338 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-08-20T06:28:41.606 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:41 smithi193 ceph-mon[21944]: Deploying daemon mon.b on smithi183 2024-08-20T06:28:41.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:41 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/1012324935' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:41.607 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:41 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:41.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:41 smithi107 ceph-mon[18495]: Deploying daemon mon.b on smithi183 2024-08-20T06:28:41.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:41 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/1012324935' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:41.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:41 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:42.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:42 smithi193 ceph-mon[21944]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:42.826 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T06:28:42.827 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T06:28:42.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:42 smithi107 ceph-mon[18495]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:43.080 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:28:43.743 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:28:43.744 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","modified":"2024-08-20T06:28:35.439375Z","created":"2024-08-20T06:27:11.020862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T06:28:43.744 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-08-20T06:28:43.763 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:43 smithi183 systemd[1]: Starting Ceph mon.b for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:28:44.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:43 smithi183 podman[22184]: 2024-08-20 06:28:43.93894878 +0000 UTC m=+0.101809482 container create a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-20T06:28:44.043 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:43 smithi183 podman[22184]: 2024-08-20 06:28:43.852951935 +0000 UTC m=+0.015812647 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:28:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 podman[22184]: 2024-08-20 06:28:44.145276234 +0000 UTC m=+0.308136942 container init a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T06:28:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 podman[22184]: 2024-08-20 06:28:44.148448372 +0000 UTC m=+0.311309075 container start a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pidfile_write: ignore empty --pid-file 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: load: jerasure load: lrc 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: RocksDB version: 7.9.2 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Git sha 0 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: DB SUMMARY 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: DB Session ID: F6HDYATKMG4ALJD5MNRO 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: CURRENT file: CURRENT 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-08-20T06:28:44.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.error_if_exists: 0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.create_if_missing: 0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.paranoid_checks: 1 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.env: 0x557f0498dbe0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.info_log: 0x557f057a9700 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.statistics: (nil) 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.use_fsync: 0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_log_file_size: 0 2024-08-20T06:28:44.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_fallocate: 1 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.use_direct_reads: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.db_log_dir: 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.wal_dir: 2024-08-20T06:28:44.489 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.write_buffer_manager: 0x557f057ace60 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T06:28:44.490 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.unordered_write: 0 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.row_cache: None 2024-08-20T06:28:44.491 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.wal_filter: None 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.two_write_queues: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.wal_compression: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.atomic_flush: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.log_readahead_size: 0 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T06:28:44.493 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_background_jobs: 2 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_background_compactions: -1 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_subcompactions: 1 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T06:28:44.494 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_open_files: -1 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_background_flushes: -1 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Compression algorithms supported: 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kZSTD supported: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kXpressCompression supported: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kBZip2Compression supported: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T06:28:44.495 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kLZ4Compression supported: 1 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kZlibCompression supported: 1 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: kSnappyCompression supported: 1 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.merge_operator: 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_filter: None 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T06:28:44.496 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557f057a9380) 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks: 1 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: pin_top_level_index_and_filter: 1 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_type: 0 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_index_type: 0 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_shortening: 1 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: checksum: 4 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: no_block_cache: 0 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache: 0x557f0579d610 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_name: BinnedLRUCache 2024-08-20T06:28:44.497 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_options: 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: capacity : 536870912 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_shard_bits : 4 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: strict_capacity_limit : 0 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: high_pri_pool_ratio: 0.000 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_cache_compressed: (nil) 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: persistent_cache: (nil) 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size: 4096 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_size_deviation: 10 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_restart_interval: 16 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: index_block_restart_interval: 1 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: metadata_block_size: 4096 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: partition_filters: 0 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: use_delta_encoding: 1 2024-08-20T06:28:44.498 INFO:journalctl@ceph.mon.b.smithi183.stdout: filter_policy: bloomfilter 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: whole_key_filtering: 1 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: verify_compression: 0 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: read_amp_bytes_per_bit: 0 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: format_version: 5 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_index_compression: 1 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: block_align: 0 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: max_auto_readahead_size: 262144 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: prepopulate_block_cache: 0 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: initial_auto_readahead_size: 8192 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression: NoCompression 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T06:28:44.499 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.num_levels: 7 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:28:44.500 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T06:28:44.501 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T06:28:44.502 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T06:28:44.503 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.inplace_update_support: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.bloom_locality: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.max_successive_merges: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.ttl: 2592000 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T06:28:44.504 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enable_blob_files: false 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.min_blob_size: 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0a1e660d-295c-4bd3-8b25-4768b71eeab5 2024-08-20T06:28:44.505 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135324175635, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135324176038, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724135324, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0a1e660d-295c-4bd3-8b25-4768b71eeab5", "db_session_id": "F6HDYATKMG4ALJD5MNRO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724135324176132, "job": 1, "event": "recovery_finished"} 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557f057cce00 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: DB pointer 0x557f058ca000 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** DB Stats ** 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:28:44.506 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.507 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** Compaction Stats [default] ** 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:28:44.508 INFO:journalctl@ceph.mon.b.smithi183.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache BinnedLRUCache@0x557f0579d610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: using public_addr v2:172.21.15.183:0/0 -> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: starting mon.b rank -1 at public addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] at bind addrs [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(???) e0 preinit fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).mds e1 new map 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).mds e1 print_map 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: e1 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: btime 2024-08-20T06:27:17:087657+0000 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T06:28:44.509 INFO:journalctl@ceph.mon.b.smithi183.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout: legacy client fscid: -1 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout: 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout: No filesystems configured 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:44.510 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi183", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Deploying cephadm binary to smithi193 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:44.511 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi183:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Added host smithi193 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.512 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/900031067' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"} : dispatch 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:44.513 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.514 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Deploying daemon mon.c on smithi193 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/2798940779' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:44.515 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.a calling monitor election 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.c calling monitor election 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: monmap epoch 2 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:44.516 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: last_changed 2024-08-20T06:28:35.439375+0000 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: min_mon_release 19 (squid) 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: election_strategy: 1 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: fsmap 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mgrmap e13: x(active, since 39s) 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: overall HEALTH_OK 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:44.517 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: Deploying daemon mon.b on smithi183 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/1012324935' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 ceph-mon[22204]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 bash[22184]: a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 2024-08-20T06:28:44.518 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:44 smithi183 systemd[1]: Started Ceph mon.b for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:28:45.234 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T06:28:45.235 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T06:28:45.487 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:28:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: mon.a calling monitor election 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: mon.c calling monitor election 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: mon.b calling monitor election 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: monmap epoch 3 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: last_changed 2024-08-20T06:28:44.201693+0000 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: min_mon_release 19 (squid) 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: election_strategy: 1 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-08-20T06:28:49.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: fsmap 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: mgrmap e13: x(active, since 48s) 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: overall HEALTH_OK 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: mon.a calling monitor election 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: mon.c calling monitor election 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: mon.b calling monitor election 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: monmap epoch 3 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: last_changed 2024-08-20T06:28:44.201693+0000 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: min_mon_release 19 (squid) 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: election_strategy: 1 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: fsmap 2024-08-20T06:28:49.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: mgrmap e13: x(active, since 48s) 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: overall HEALTH_OK 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: mon.a calling monitor election 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: mon.c calling monitor election 2024-08-20T06:28:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: mon.b calling monitor election 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: monmap epoch 3 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: last_changed 2024-08-20T06:28:44.201693+0000 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: created 2024-08-20T06:27:11.020862+0000 2024-08-20T06:28:49.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: min_mon_release 19 (squid) 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: election_strategy: 1 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: 0: [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon.a 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.c 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: 2: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: fsmap 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: mgrmap e13: x(active, since 48s) 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: overall HEALTH_OK 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:49.814 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:28:49.814 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":3,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","modified":"2024-08-20T06:28:44.201693Z","created":"2024-08-20T06:27:11.020862Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-20T06:28:49.814 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 3 2024-08-20T06:28:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:50 smithi193 ceph-mon[21944]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:50 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/2427828888' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:50.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:50 smithi107 ceph-mon[18495]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:50.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:50.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:50 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/2427828888' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:50.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:50.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:50 smithi183 ceph-mon[22204]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:50 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/2427828888' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T06:28:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:28:50.970 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-20T06:28:50.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-20T06:28:51.218 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:51.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi107:/etc/ceph/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:51 smithi107 ceph-mon[18495]: Reconfiguring daemon mon.a on smithi107 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi107:/etc/ceph/ceph.conf 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T06:28:51.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:51.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:51.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:51 smithi183 ceph-mon[22204]: Reconfiguring daemon mon.a on smithi107 2024-08-20T06:28:51.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi107:/etc/ceph/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi183:/etc/ceph/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi193:/etc/ceph/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Updating smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:51.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:51.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:51 smithi193 ceph-mon[21944]: Reconfiguring daemon mon.a on smithi107 2024-08-20T06:28:51.871 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:51.871 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-08-20T06:28:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:28:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-08-20T06:28:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:52 smithi193 ceph-mon[21944]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:52 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4905374' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:52 smithi107 ceph-mon[18495]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4905374' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:52.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:52 smithi183 ceph-mon[22204]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:52 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4905374' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:53.368 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-20T06:28:53.368 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:53.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T06:28:53.398 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:28:53.398 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:53.465 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:28:53.465 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T06:28:53.493 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:28:53.493 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:53.560 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:28:53.560 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T06:28:53.589 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:28:53.589 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T06:28:53.655 INFO:tasks.cephadm:Adding mgr.x on smithi107 2024-08-20T06:28:53.655 INFO:tasks.cephadm:Adding mgr.y on smithi183 2024-08-20T06:28:53.655 INFO:tasks.cephadm:Adding mgr.z on smithi193 2024-08-20T06:28:53.655 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi107=x;smithi183=y;smithi193=z' 2024-08-20T06:28:53.934 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: Reconfiguring mon.b (monmap changed)... 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:54.234 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:53 smithi183 ceph-mon[22204]: Reconfiguring daemon mon.b on smithi183 2024-08-20T06:28:54.285 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: Reconfiguring mon.b (monmap changed)... 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:54.286 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:53 smithi193 ceph-mon[21944]: Reconfiguring daemon mon.b on smithi183 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: Reconfiguring mon.b (monmap changed)... 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:54.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:54.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:53 smithi107 ceph-mon[18495]: Reconfiguring daemon mon.b on smithi183 2024-08-20T06:28:54.585 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mgr update... 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:55.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:55.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:55.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.106 DEBUG:teuthology.orchestra.run.smithi183:mgr.y> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.y.service 2024-08-20T06:28:56.109 DEBUG:teuthology.orchestra.run.smithi193:mgr.z> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.z.service 2024-08-20T06:28:56.111 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-20T06:28:56.111 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-20T06:28:56.114 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-20T06:28:56.114 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set client debug_client 20 2024-08-20T06:28:56.158 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-20T06:28:56.158 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set client debug_ms 1 2024-08-20T06:28:56.203 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-20T06:28:56.203 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-20T06:28:56.245 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-20T06:28:56.246 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-20T06:28:56.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: Reconfiguring mon.c (monmap changed)... 2024-08-20T06:28:56.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: Reconfiguring daemon mon.c on smithi193 2024-08-20T06:28:56.265 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: Saving service mgr spec with placement smithi107=x;smithi183=y;smithi193=z;count:3 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:56.266 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.267 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-20T06:28:56.267 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: Reconfiguring mon.c (monmap changed)... 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: Reconfiguring daemon mon.c on smithi193 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: Saving service mgr spec with placement smithi107=x;smithi183=y;smithi193=z;count:3 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.289 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:56.290 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.310 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-20T06:28:56.311 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-20T06:28:56.354 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-20T06:28:56.354 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-20T06:28:56.397 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-20T06:28:56.397 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-20T06:28:56.402 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:56.406 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-20T06:28:56.407 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-20T06:28:56.450 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-20T06:28:56.451 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-20T06:28:56.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: Reconfiguring mon.c (monmap changed)... 2024-08-20T06:28:56.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: Reconfiguring daemon mon.c on smithi193 2024-08-20T06:28:56.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi107=x;smithi183=y;smithi193=z", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: Saving service mgr spec with placement smithi107=x;smithi183=y;smithi193=z;count:3 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:56.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:56.493 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-20T06:28:56.494 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-20T06:28:56.549 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-20T06:28:56.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-20T06:28:56.592 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-20T06:28:56.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-20T06:28:56.637 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-20T06:28:56.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-20T06:28:56.680 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-20T06:28:56.681 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-20T06:28:56.729 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-20T06:28:56.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-20T06:28:56.775 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-20T06:28:56.776 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-20T06:28:56.824 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-20T06:28:56.824 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-20T06:28:56.877 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-20T06:28:56.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-20T06:28:56.922 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-20T06:28:56.922 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-20T06:28:56.971 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-20T06:28:56.971 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-20T06:28:57.016 DEBUG:tasks.cephadm:set 22 configs 2024-08-20T06:28:57.016 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi107') 2024-08-20T06:28:57.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:56 smithi107 ceph-mon[18495]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:57.046 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:56 smithi107 ceph-mon[18495]: Deploying daemon mgr.z on smithi193 2024-08-20T06:28:57.256 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:56 smithi193 ceph-mon[21944]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:57.257 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:56 smithi193 ceph-mon[21944]: Deploying daemon mgr.z on smithi193 2024-08-20T06:28:57.294 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:56 smithi183 ceph-mon[22204]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:56 smithi183 ceph-mon[22204]: Deploying daemon mgr.z on smithi193 2024-08-20T06:28:57.554 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 systemd[1]: Starting Ceph mgr.z for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:28:57.900 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 podman[23508]: 2024-08-20 06:28:57.542413987 +0000 UTC m=+0.016090395 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:28:57.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 podman[23508]: 2024-08-20 06:28:57.662366776 +0000 UTC m=+0.136043224 container create b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731) 2024-08-20T06:28:57.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 podman[23508]: 2024-08-20 06:28:57.87276326 +0000 UTC m=+0.346439665 container init b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T06:28:57.901 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 podman[23508]: 2024-08-20 06:28:57.875439601 +0000 UTC m=+0.349116013 container start b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T06:28:57.953 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:57.953 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:57.953 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:57.993 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:28:58.304 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 ceph-mgr[23527]: -- 172.21.15.193:0/3110134471 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556842e570e0 con 0x556842e35400 2024-08-20T06:28:58.304 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 bash[23508]: b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 2024-08-20T06:28:58.304 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:28:57 smithi193 systemd[1]: Started Ceph mgr.z for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:28:58.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='client.? ' entity='client.admin' 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: Deploying daemon mgr.y on smithi183 2024-08-20T06:28:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:28:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1241862008' entity='client.admin' 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='client.? ' entity='client.admin' 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:58.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: Deploying daemon mgr.y on smithi183 2024-08-20T06:28:58.744 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:28:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1241862008' entity='client.admin' 2024-08-20T06:28:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='client.? ' entity='client.admin' 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:28:58.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: Deploying daemon mgr.y on smithi183 2024-08-20T06:28:58.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:28:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1241862008' entity='client.admin' 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd36003ed60 msgr2=0x7fd360041220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd36003ed60 0x7fd360041220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd368009c20 tx=0x7fd368009050 comp rx=0 tx=0).stop 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd378102770 msgr2=0x7fd378198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd378102770 0x7fd378198e50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00d950 tx=0x7fd36c00de10 comp rx=0 tx=0).stop 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 shutdown_connections 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd36003ed60 0x7fd360041220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd378103130 0x7fd378199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd378102770 0x7fd378198e50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 --2- 172.21.15.107:0/3729150609 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd378101610 0x7fd378198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 >> 172.21.15.107:0/3729150609 conn(0x7fd3780fcee0 msgr2=0x7fd37806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 shutdown_connections 2024-08-20T06:28:59.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.023+0000 7fd37d461640 1 -- 172.21.15.107:0/3729150609 wait complete. 2024-08-20T06:28:59.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3423944014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 msgr2=0x7fc904106700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3423944014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc904106700 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009ca0 tx=0x7fc8fc02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3423944014 shutdown_connections 2024-08-20T06:28:59.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3423944014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc904106700 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3423944014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc9041036a0 0x7fc904105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3423944014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc90406bdc0 0x7fc904103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3423944014 >> 172.21.15.107:0/3423944014 conn(0x7fc9040fcf30 msgr2=0x7fc9040ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3423944014 shutdown_connections 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.126+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3423944014 wait complete. 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 Processor -- start 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 -- start start 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90406bdc0 0x7fc90419ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9041036a0 0x7fc90419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc9041145c0 con 0x7fc904106250 2024-08-20T06:28:59.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc904114440 con 0x7fc90406bdc0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90b3ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc904114740 con 0x7fc9041036a0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39480/0 (socket says 172.21.15.107:39480) 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 -- 172.21.15.107:0/3311732314 learned_addr learned my addr 172.21.15.107:0/3311732314 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc908988640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9041036a0 0x7fc90419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc909189640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90406bdc0 0x7fc90419ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 -- 172.21.15.107:0/3311732314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9041036a0 msgr2=0x7fc90419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9041036a0 0x7fc90419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 -- 172.21.15.107:0/3311732314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90406bdc0 msgr2=0x7fc90419ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90406bdc0 0x7fc90419ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 -- 172.21.15.107:0/3311732314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9041a0990 con 0x7fc904106250 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.127+0000 7fc90998a640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:28:59.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.128+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc004040 con 0x7fc904106250 2024-08-20T06:28:59.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.128+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fc8fc004630 con 0x7fc904106250 2024-08-20T06:28:59.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.128+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9041a0c20 con 0x7fc904106250 2024-08-20T06:28:59.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.128+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8fc030d00 con 0x7fc904106250 2024-08-20T06:28:59.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.128+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9041a7360 con 0x7fc904106250 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.129+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 50748+0+0 (secure 0 0 0) 0x7fc8fc0041e0 con 0x7fc904106250 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.129+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc904106700 con 0x7fc904106250 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.129+0000 7fc8fa7fc640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 0x7fc8f0041220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.129+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc8fc074960 con 0x7fc904106250 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.129+0000 7fc909189640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 0x7fc8f0041220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:28:59.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.130+0000 7fc909189640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 0x7fc8f0041220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc900005d00 tx=0x7fc900005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:28:59.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.136+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc8fc0305c0 con 0x7fc904106250 2024-08-20T06:28:59.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.314+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fc90402e590 con 0x7fc904106250 2024-08-20T06:28:59.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.317+0000 7fc8fa7fc640 1 -- 172.21.15.107:0/3311732314 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v13)=0 v13) ==== 106+0+0 (secure 0 0 0) 0x7fc8fc03b410 con 0x7fc904106250 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 msgr2=0x7fc8f0041220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 0x7fc8f0041220 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc900005d00 tx=0x7fc900005c50 comp rx=0 tx=0).stop 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 msgr2=0x7fc9041a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc006010 tx=0x7fc8fc003ba0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 shutdown_connections 2024-08-20T06:28:59.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc904106250 0x7fc9041a00f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc8f003ed60 0x7fc8f0041220 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9041036a0 0x7fc90419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 --2- 172.21.15.107:0/3311732314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc90406bdc0 0x7fc90419ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 >> 172.21.15.107:0/3311732314 conn(0x7fc9040fcf30 msgr2=0x7fc904101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 shutdown_connections 2024-08-20T06:28:59.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:28:59.320+0000 7fc90b3ed640 1 -- 172.21.15.107:0/3311732314 wait complete. 2024-08-20T06:29:00.133 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:00 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3729150609' entity='client.admin' 2024-08-20T06:29:00.134 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:00 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3311732314' entity='client.admin' 2024-08-20T06:29:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:00 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3729150609' entity='client.admin' 2024-08-20T06:29:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:00 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3311732314' entity='client.admin' 2024-08-20T06:29:00.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:00 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3729150609' entity='client.admin' 2024-08-20T06:29:00.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:00 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3311732314' entity='client.admin' 2024-08-20T06:29:00.486 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Aug 20 06:29:00 smithi183 systemd[1]: Started Ceph mgr.y for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:29:01.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 -- 172.21.15.107:0/3791535554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c102770 msgr2=0x7f111c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:01.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 --2- 172.21.15.107:0/3791535554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c102770 0x7f111c102bf0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1110009930 tx=0x7f111002a420 comp rx=0 tx=0).stop 2024-08-20T06:29:01.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 -- 172.21.15.107:0/3791535554 shutdown_connections 2024-08-20T06:29:01.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 --2- 172.21.15.107:0/3791535554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c103130 0x7f111c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 --2- 172.21.15.107:0/3791535554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c102770 0x7f111c102bf0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 --2- 172.21.15.107:0/3791535554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c101610 0x7f111c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 -- 172.21.15.107:0/3791535554 >> 172.21.15.107:0/3791535554 conn(0x7f111c0fcee0 msgr2=0x7f111c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 -- 172.21.15.107:0/3791535554 shutdown_connections 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.106+0000 7f1120d31640 1 -- 172.21.15.107:0/3791535554 wait complete. 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 Processor -- start 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 -- start start 2024-08-20T06:29:01.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c102770 0x7f111c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c103130 0x7f111c199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10cf10 con 0x7f111c101610 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10cd90 con 0x7f111c102770 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1120d31640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f111c10d090 con 0x7f111c103130 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f111a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f111a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39506/0 (socket says 172.21.15.107:39506) 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f111a59c640 1 -- 172.21.15.107:0/635311123 learned_addr learned my addr 172.21.15.107:0/635311123 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f1119d9b640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c102770 0x7f111c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.107+0000 7f111ad9d640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c103130 0x7f111c199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:01.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 -- 172.21.15.107:0/635311123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c103130 msgr2=0x7f111c199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c103130 0x7f111c199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 -- 172.21.15.107:0/635311123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c102770 msgr2=0x7f111c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c102770 0x7f111c198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 -- 172.21.15.107:0/635311123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111c19a3b0 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f111a59c640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d770 tx=0x7f110c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c017070 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7f110c005470 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110c004a70 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f111c19a6a0 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.108+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f111c1a0d80 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.109+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) ==== 50748+0+0 (secure 0 0 0) 0x7f110c004c10 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.109+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f111c101a10 con 0x7f111c101610 2024-08-20T06:29:01.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.109+0000 7f110b7fe640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 0x7f1104041220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:01.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.109+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f110c052ce0 con 0x7f111c101610 2024-08-20T06:29:01.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.109+0000 7f1119d9b640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 0x7f1104041220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:01.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.110+0000 7f1119d9b640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 0x7f1104041220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1110004770 tx=0x7f1110000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:01.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.114+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f110c009c30 con 0x7f111c101610 2024-08-20T06:29:01.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.296+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f111c1a1070 con 0x7f111c101610 2024-08-20T06:29:01.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:01.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.297 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.298 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:01.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.301+0000 7f110b7fe640 1 -- 172.21.15.107:0/635311123 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v14)=0 v14) ==== 108+0+0 (secure 0 0 0) 0x7f110c00ed40 con 0x7f111c101610 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 msgr2=0x7f1104041220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 0x7f1104041220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1110004770 tx=0x7f1110000bc0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 msgr2=0x7f111c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f110c00d770 tx=0x7f110c00dc30 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 shutdown_connections 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f110403ed60 0x7f1104041220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c103130 0x7f111c199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f111c102770 0x7f111c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 --2- 172.21.15.107:0/635311123 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f111c101610 0x7f111c1988f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 >> 172.21.15.107:0/635311123 conn(0x7f111c0fcee0 msgr2=0x7f111c06a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 shutdown_connections 2024-08-20T06:29:01.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:01.304+0000 7f1120d31640 1 -- 172.21.15.107:0/635311123 wait complete. 2024-08-20T06:29:01.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:01.571 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.572 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.576 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.578 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.581 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.581 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.582 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.582 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.587 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.667 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi107') 2024-08-20T06:29:01.667 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi107') 2024-08-20T06:29:01.667 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi107') 2024-08-20T06:29:01.866 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:01.869 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:02.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/635311123' entity='client.admin' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:29:02.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:29:02.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:02 smithi107 ceph-mon[18495]: Reconfiguring daemon mgr.x on smithi107 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/635311123' entity='client.admin' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:02 smithi183 ceph-mon[22204]: Reconfiguring daemon mgr.x on smithi107 2024-08-20T06:29:02.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/635311123' entity='client.admin' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T06:29:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:02.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:02 smithi193 ceph-mon[21944]: Reconfiguring daemon mgr.x on smithi107 2024-08-20T06:29:03.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 -- 172.21.15.107:0/489282543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808103130 msgr2=0x7fd808109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 --2- 172.21.15.107:0/489282543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808103130 0x7fd808109ab0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd800009cd0 tx=0x7fd80002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 -- 172.21.15.107:0/489282543 shutdown_connections 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 --2- 172.21.15.107:0/489282543 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808103130 0x7fd808109ab0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 --2- 172.21.15.107:0/489282543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808102770 0x7fd808102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 --2- 172.21.15.107:0/489282543 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808101610 0x7fd808101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 -- 172.21.15.107:0/489282543 >> 172.21.15.107:0/489282543 conn(0x7fd8080fcee0 msgr2=0x7fd8080ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 -- 172.21.15.107:0/489282543 shutdown_connections 2024-08-20T06:29:03.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.492+0000 7fd80f70e640 1 -- 172.21.15.107:0/489282543 wait complete. 2024-08-20T06:29:03.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 Processor -- start 2024-08-20T06:29:03.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 -- start start 2024-08-20T06:29:03.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808101610 0x7fd80806b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:03.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 0x7fd80819fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810cec0 con 0x7fd808102770 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810cd40 con 0x7fd808103130 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.493+0000 7fd80f70e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd80810d040 con 0x7fd808101610 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80cca9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 0x7fd80819fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80cca9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39526/0 (socket says 172.21.15.107:39526) 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80cca9640 1 -- 172.21.15.107:0/3800817542 learned_addr learned my addr 172.21.15.107:0/3800817542 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80d4aa640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808101610 0x7fd80806b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 -- 172.21.15.107:0/3800817542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808101610 msgr2=0x7fd80806b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808101610 0x7fd80806b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 -- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 msgr2=0x7fd80819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 -- 172.21.15.107:0/3800817542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8081a03f0 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80cca9640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80dcab640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 0x7fd80819fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd800031040 tx=0x7fd800003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800004240 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fd800004830 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8081a0680 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.494+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8081a0bc0 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.495+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd800030c20 con 0x7fd808103130 2024-08-20T06:29:03.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.495+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7ec0050b0 con 0x7fd808103130 2024-08-20T06:29:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.496+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 14) ==== 100876+0+0 (secure 0 0 0) 0x7fd8000043e0 con 0x7fd808103130 2024-08-20T06:29:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.497+0000 7fd7fe7fc640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 0x7fd7f407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.497+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd80002e030 con 0x7fd808103130 2024-08-20T06:29:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.497+0000 7fd80d4aa640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 0x7fd7f407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:03.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.497+0000 7fd80d4aa640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 0x7fd7f407b910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd804009bd0 tx=0x7fd804005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:03.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.500+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd80004b060 con 0x7fd808103130 2024-08-20T06:29:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:03 smithi107 ceph-mon[18495]: Standby manager daemon z started 2024-08-20T06:29:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:03 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T06:29:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:03 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:03 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T06:29:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:03 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:03.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.680+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fd7ec0056a0 con 0x7fd808103130 2024-08-20T06:29:03.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.685+0000 7fd7fe7fc640 1 -- 172.21.15.107:0/3800817542 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v15)=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7fd800084fb0 con 0x7fd808103130 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 msgr2=0x7fd7f407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 0x7fd7f407b910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd804009bd0 tx=0x7fd804005f90 comp rx=0 tx=0).stop 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 msgr2=0x7fd80819fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 0x7fd80819fb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd800031040 tx=0x7fd800003ba0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 shutdown_connections 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd808103130 0x7fd80819fb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd808102770 0x7fd80819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd7f4079450 0x7fd7f407b910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 --2- 172.21.15.107:0/3800817542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd808101610 0x7fd80806b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:03.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.687+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 >> 172.21.15.107:0/3800817542 conn(0x7fd8080fcee0 msgr2=0x7fd808105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:03.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.688+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 shutdown_connections 2024-08-20T06:29:03.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:03.688+0000 7fd80f70e640 1 -- 172.21.15.107:0/3800817542 wait complete. 2024-08-20T06:29:03.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:03 smithi183 ceph-mon[22204]: Standby manager daemon z started 2024-08-20T06:29:03.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:03 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T06:29:03.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:03 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:03 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T06:29:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:03 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:03 smithi193 ceph-mon[21944]: Standby manager daemon z started 2024-08-20T06:29:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:03 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T06:29:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:03 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:03 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T06:29:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:03 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.193:0/2067104118' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:04.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 -- 172.21.15.107:0/4020604769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320103130 msgr2=0x7fd320109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 --2- 172.21.15.107:0/4020604769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320103130 0x7fd320109ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd318009cd0 tx=0x7fd31802a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 -- 172.21.15.107:0/4020604769 shutdown_connections 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 --2- 172.21.15.107:0/4020604769 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320103130 0x7fd320109ab0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 --2- 172.21.15.107:0/4020604769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320102770 0x7fd320102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 --2- 172.21.15.107:0/4020604769 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320101610 0x7fd320101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 -- 172.21.15.107:0/4020604769 >> 172.21.15.107:0/4020604769 conn(0x7fd3200fcee0 msgr2=0x7fd3200ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 -- 172.21.15.107:0/4020604769 shutdown_connections 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.299+0000 7fd3285ce640 1 -- 172.21.15.107:0/4020604769 wait complete. 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 Processor -- start 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 -- start start 2024-08-20T06:29:04.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320101610 0x7fd320197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320103130 0x7fd3201989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010cef0 con 0x7fd320102770 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010cd70 con 0x7fd320103130 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd3285ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd32010d070 con 0x7fd320101610 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39534/0 (socket says 172.21.15.107:39534) 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 -- 172.21.15.107:0/1357888647 learned_addr learned my addr 172.21.15.107:0/1357888647 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd326b6b640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320103130 0x7fd3201989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd32636a640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320101610 0x7fd320197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 -- 172.21.15.107:0/1357888647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320101610 msgr2=0x7fd320197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320101610 0x7fd320197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 -- 172.21.15.107:0/1357888647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320103130 msgr2=0x7fd3201989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320103130 0x7fd3201989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 -- 172.21.15.107:0/1357888647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd320199240 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.300+0000 7fd325b69640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd31000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.301+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310017070 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.301+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fd310009c30 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.301+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31000f3c0 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.301+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd320199530 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.301+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32019fc10 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.302+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd320101a10 con 0x7fd320102770 2024-08-20T06:29:04.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.302+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 100876+0+0 (secure 0 0 0) 0x7fd310010460 con 0x7fd320102770 2024-08-20T06:29:04.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.303+0000 7fd3177fe640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 0x7fd30c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.303+0000 7fd32636a640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 0x7fd30c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.303+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd31009a3e0 con 0x7fd320102770 2024-08-20T06:29:04.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.303+0000 7fd32636a640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 0x7fd30c07b910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00bbe0 tx=0x7fd31c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.307+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd310063250 con 0x7fd320102770 2024-08-20T06:29:04.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 -- 172.21.15.107:0/414287763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08103130 msgr2=0x7fad08109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/414287763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08103130 0x7fad08109ab0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fad00009cd0 tx=0x7fad0002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 -- 172.21.15.107:0/414287763 shutdown_connections 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/414287763 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08103130 0x7fad08109ab0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/414287763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08102770 0x7fad08102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/414287763 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08101610 0x7fad08101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 -- 172.21.15.107:0/414287763 >> 172.21.15.107:0/414287763 conn(0x7fad080fcee0 msgr2=0x7fad080ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.486+0000 7fad0e8f0640 1 -- 172.21.15.107:0/414287763 shutdown_connections 2024-08-20T06:29:04.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:04 smithi107 ceph-mon[18495]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:04.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:04 smithi107 ceph-mon[18495]: mgrmap e14: x(active, since 62s), standbys: z 2024-08-20T06:29:04.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:04 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T06:29:04.489 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:04 smithi107 ceph-mon[18495]: from='client.? ' entity='client.admin' 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.487+0000 7fad0e8f0640 1 -- 172.21.15.107:0/414287763 wait complete. 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 Processor -- start 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 -- start start 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08101610 0x7fad081053d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08103130 0x7fad08104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810cea0 con 0x7fad08102770 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810cd20 con 0x7fad08103130 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0e8f0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fad0810d020 con 0x7fad08101610 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad077fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad077fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39548/0 (socket says 172.21.15.107:39548) 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad077fe640 1 -- 172.21.15.107:0/3957010639 learned_addr learned my addr 172.21.15.107:0/3957010639 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:04.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad0ce8d640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08103130 0x7fad08104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.488+0000 7fad07fff640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08101610 0x7fad081053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 -- 172.21.15.107:0/3957010639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08101610 msgr2=0x7fad081053d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08101610 0x7fad081053d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 -- 172.21.15.107:0/3957010639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08103130 msgr2=0x7fad08104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08103130 0x7fad08104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 -- 172.21.15.107:0/3957010639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad08104a40 con 0x7fad08102770 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.489+0000 7fad077fe640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bec0 tx=0x7facfc00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.490+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc028030 con 0x7fad08102770 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.490+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7facfc004020 con 0x7fad08102770 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.490+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad081984a0 con 0x7fad08102770 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.490+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facfc005450 con 0x7fad08102770 2024-08-20T06:29:04.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.490+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad08198a30 con 0x7fad08102770 2024-08-20T06:29:04.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.491+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad08101a10 con 0x7fad08102770 2024-08-20T06:29:04.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.492+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 100876+0+0 (secure 0 0 0) 0x7facfc009a30 con 0x7fad08102770 2024-08-20T06:29:04.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.492+0000 7fad057fa640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 0x7facf007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.492+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fd320063a80 con 0x7fd320102770 2024-08-20T06:29:04.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.492+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7facfc0ab130 con 0x7fad08102770 2024-08-20T06:29:04.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.492+0000 7fad07fff640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 0x7facf007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.493+0000 7fad07fff640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 0x7facf007b910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7facf8009bd0 tx=0x7facf8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.497+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7facfc073fa0 con 0x7fad08102770 2024-08-20T06:29:04.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.502+0000 7fd3177fe640 1 -- 172.21.15.107:0/1357888647 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v16)=0 v16) ==== 116+0+0 (secure 0 0 0) 0x7fd3100677a0 con 0x7fd320102770 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 msgr2=0x7fd30c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 0x7fd30c07b910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00bbe0 tx=0x7fd31c00b390 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 msgr2=0x7fd320197d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd31000de10 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 shutdown_connections 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd320103130 0x7fd3201989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd320102770 0x7fd320197d60 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd31000d950 tx=0x7fd31000de10 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd30c079450 0x7fd30c07b910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 --2- 172.21.15.107:0/1357888647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd320101610 0x7fd320197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 >> 172.21.15.107:0/1357888647 conn(0x7fd3200fcee0 msgr2=0x7fd320105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 shutdown_connections 2024-08-20T06:29:04.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.505+0000 7fd3285ce640 1 -- 172.21.15.107:0/1357888647 wait complete. 2024-08-20T06:29:04.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.684+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fad08199320 con 0x7fad08102770 2024-08-20T06:29:04.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.689+0000 7fad057fa640 1 -- 172.21.15.107:0/3957010639 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7facfc0784f0 con 0x7fad08102770 2024-08-20T06:29:04.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 msgr2=0x7facf007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 0x7facf007b910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7facf8009bd0 tx=0x7facf8005f90 comp rx=0 tx=0).stop 2024-08-20T06:29:04.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 msgr2=0x7fad08103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facfc00bec0 tx=0x7facfc00dd20 comp rx=0 tx=0).stop 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 shutdown_connections 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fad08103130 0x7fad08104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fad08102770 0x7fad08103c40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7facf0079450 0x7facf007b910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 --2- 172.21.15.107:0/3957010639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad08101610 0x7fad081053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.691+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 >> 172.21.15.107:0/3957010639 conn(0x7fad080fcee0 msgr2=0x7fad0806a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.692+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 shutdown_connections 2024-08-20T06:29:04.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.692+0000 7fad0e8f0640 1 -- 172.21.15.107:0/3957010639 wait complete. 2024-08-20T06:29:04.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.724+0000 7fd276dda640 1 -- 172.21.15.107:0/602473797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27006b340 msgr2=0x7fd270105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- 172.21.15.107:0/602473797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27006b340 0x7fd270105390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd264009900 tx=0x7fd26402a410 comp rx=0 tx=0).stop 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- 172.21.15.107:0/602473797 shutdown_connections 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- 172.21.15.107:0/602473797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd270108420 0x7fd27010a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- 172.21.15.107:0/602473797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- 172.21.15.107:0/602473797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd27006b340 0x7fd270105390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- 172.21.15.107:0/602473797 >> 172.21.15.107:0/602473797 conn(0x7fd2700fcee0 msgr2=0x7fd2700ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- 172.21.15.107:0/602473797 shutdown_connections 2024-08-20T06:29:04.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- 172.21.15.107:0/602473797 wait complete. 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 Processor -- start 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- start start 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 0x7fd270102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270101330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd27010e000 con 0x7fd270108420 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd27010de80 con 0x7fd2701058d0 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.725+0000 7fd276dda640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd27010e180 con 0x7fd27006b340 2024-08-20T06:29:04.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270101330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39558/0 (socket says 172.21.15.107:39558) 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd26ffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270101330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:35132/0 (socket says 172.21.15.107:35132) 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 -- 172.21.15.107:0/1387284980 learned_addr learned my addr 172.21.15.107:0/1387284980 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd274b76640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 0x7fd270102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 -- 172.21.15.107:0/1387284980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 msgr2=0x7fd270102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 0x7fd270102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 -- 172.21.15.107:0/1387284980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 msgr2=0x7fd270101330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270101330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 -- 172.21.15.107:0/1387284980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd270102130 con 0x7fd270108420 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd274b76640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 0x7fd270102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd275377640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd26800b970 tx=0x7fd26800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268019070 con 0x7fd270108420 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(23 keys) ==== 1001+0+0 (secure 0 0 0) 0x7fd268014440 con 0x7fd270108420 2024-08-20T06:29:04.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2701a2960 con 0x7fd270108420 2024-08-20T06:29:04.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.726+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2680075d0 con 0x7fd270108420 2024-08-20T06:29:04.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.727+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701a2ef0 con 0x7fd270108420 2024-08-20T06:29:04.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.728+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd270105cd0 con 0x7fd270108420 2024-08-20T06:29:04.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.728+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) ==== 100876+0+0 (secure 0 0 0) 0x7fd26801c070 con 0x7fd270108420 2024-08-20T06:29:04.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.729+0000 7fd26dffb640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 0x7fd26007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:04.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.729+0000 7fd274b76640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 0x7fd26007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:04.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.729+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd26809a060 con 0x7fd270108420 2024-08-20T06:29:04.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.729+0000 7fd274b76640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 0x7fd26007b910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:04.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.733+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd268062ed0 con 0x7fd270108420 2024-08-20T06:29:04.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:04 smithi183 ceph-mon[22204]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:04 smithi183 ceph-mon[22204]: mgrmap e14: x(active, since 62s), standbys: z 2024-08-20T06:29:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:04 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T06:29:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:04 smithi183 ceph-mon[22204]: from='client.? ' entity='client.admin' 2024-08-20T06:29:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:04 smithi193 ceph-mon[21944]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:04 smithi193 ceph-mon[21944]: mgrmap e14: x(active, since 62s), standbys: z 2024-08-20T06:29:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:04 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T06:29:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:04 smithi193 ceph-mon[21944]: from='client.? ' entity='client.admin' 2024-08-20T06:29:04.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.910+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fd27002e4f0 con 0x7fd270108420 2024-08-20T06:29:04.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.913+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v18)=0 v18) ==== 123+0+0 (secure 0 0 0) 0x7fd268067420 con 0x7fd270108420 2024-08-20T06:29:04.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.914+0000 7fd26dffb640 1 -- 172.21.15.107:0/1387284980 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fd268014e50 con 0x7fd270108420 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 msgr2=0x7fd26007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 0x7fd26007b910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd264000c00 tx=0x7fd264005a20 comp rx=0 tx=0).stop 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 msgr2=0x7fd270101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd26800b970 tx=0x7fd26800be30 comp rx=0 tx=0).stop 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 shutdown_connections 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd270108420 0x7fd270101870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2701058d0 0x7fd270101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd260079450 0x7fd26007b910 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 --2- 172.21.15.107:0/1387284980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd27006b340 0x7fd270102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:04.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 >> 172.21.15.107:0/1387284980 conn(0x7fd2700fcee0 msgr2=0x7fd270103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:04.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 shutdown_connections 2024-08-20T06:29:04.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:04.916+0000 7fd276dda640 1 -- 172.21.15.107:0/1387284980 wait complete. 2024-08-20T06:29:05.505 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi107') 2024-08-20T06:29:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1357888647' entity='client.admin' 2024-08-20T06:29:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3957010639' entity='client.admin' 2024-08-20T06:29:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1387284980' entity='client.admin' 2024-08-20T06:29:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: Standby manager daemon y started 2024-08-20T06:29:05.796 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T06:29:05.797 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:05.797 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T06:29:05.797 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:05 smithi107 ceph-mon[18495]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1357888647' entity='client.admin' 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3957010639' entity='client.admin' 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1387284980' entity='client.admin' 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: Standby manager daemon y started 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T06:29:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:05 smithi193 ceph-mon[21944]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1357888647' entity='client.admin' 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3957010639' entity='client.admin' 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1387284980' entity='client.admin' 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: Standby manager daemon y started 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T06:29:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T06:29:05.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:05 smithi183 ceph-mon[22204]: from='mgr.? 172.21.15.183:0/356555423' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.998+0000 7f55c150b640 1 -- 172.21.15.107:0/3568002990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc102760 msgr2=0x7f55bc102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.998+0000 7f55c150b640 1 --2- 172.21.15.107:0/3568002990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc102760 0x7f55bc102b60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009cd0 tx=0x7f55ac02a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.998+0000 7f55c150b640 1 -- 172.21.15.107:0/3568002990 shutdown_connections 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.998+0000 7f55c150b640 1 --2- 172.21.15.107:0/3568002990 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc104280 0x7f55bc10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.998+0000 7f55c150b640 1 --2- 172.21.15.107:0/3568002990 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc1038c0 0x7f55bc103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 --2- 172.21.15.107:0/3568002990 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc102760 0x7f55bc102b60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- 172.21.15.107:0/3568002990 >> 172.21.15.107:0/3568002990 conn(0x7f55bc0fe050 msgr2=0x7f55bc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- 172.21.15.107:0/3568002990 shutdown_connections 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- 172.21.15.107:0/3568002990 wait complete. 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 Processor -- start 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- start start 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 0x7f55bc1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc104280 0x7f55bc199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10e200 con 0x7f55bc1038c0 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10e080 con 0x7f55bc104280 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:05.999+0000 7f55c150b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc10e380 con 0x7f55bc102760 2024-08-20T06:29:06.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42838/0 (socket says 172.21.15.107:42838) 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 -- 172.21.15.107:0/3928543514 learned_addr learned my addr 172.21.15.107:0/3928543514 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55bb7fe640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc104280 0x7f55bc199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55baffd640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 0x7f55bc1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 -- 172.21.15.107:0/3928543514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 msgr2=0x7f55bc1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 0x7f55bc1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 -- 172.21.15.107:0/3928543514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc104280 msgr2=0x7f55bc199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc104280 0x7f55bc199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 -- 172.21.15.107:0/3928543514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc19a4f0 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55baffd640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 0x7f55bc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55ba7fc640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d950 tx=0x7f55a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a4017070 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f55a4009c30 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f620 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc19a7e0 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.000+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a0ec0 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.001+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc109b00 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.002+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f55a4010460 con 0x7f55bc1038c0 2024-08-20T06:29:06.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.003+0000 7f55b3fff640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 0x7f55a80b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.003+0000 7f55baffd640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 0x7f55a80b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.003+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f55a40e1b40 con 0x7f55bc1038c0 2024-08-20T06:29:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.004+0000 7f55baffd640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 0x7f55a80b6000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55ac031590 tx=0x7f55ac000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:06.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.007+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f55a40aae60 con 0x7f55bc1038c0 2024-08-20T06:29:06.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.187+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f55bc02e340 con 0x7f55bc1038c0 2024-08-20T06:29:06.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.191+0000 7f55b3fff640 1 -- 172.21.15.107:0/3928543514 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f55a4009de0 con 0x7f55bc1038c0 2024-08-20T06:29:06.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 msgr2=0x7f55a80b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:06.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 0x7f55a80b6000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f55ac031590 tx=0x7f55ac000f40 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 msgr2=0x7f55bc199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f55a400d950 tx=0x7f55a400de10 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 shutdown_connections 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.193+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f55bc104280 0x7f55bc199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55bc1038c0 0x7f55bc199010 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f55a80b3b40 0x7f55a80b6000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 --2- 172.21.15.107:0/3928543514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f55bc102760 0x7f55bc1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 >> 172.21.15.107:0/3928543514 conn(0x7f55bc0fe050 msgr2=0x7f55bc106cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 shutdown_connections 2024-08-20T06:29:06.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:06.194+0000 7f55c150b640 1 -- 172.21.15.107:0/3928543514 wait complete. 2024-08-20T06:29:07.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:06 smithi107 ceph-mon[18495]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:07.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:06 smithi107 ceph-mon[18495]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-08-20T06:29:07.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:06 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T06:29:07.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:06 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3928543514' entity='client.admin' 2024-08-20T06:29:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:06 smithi183 ceph-mon[22204]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:06 smithi183 ceph-mon[22204]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-08-20T06:29:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:06 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T06:29:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:06 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3928543514' entity='client.admin' 2024-08-20T06:29:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:06 smithi193 ceph-mon[21944]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:06 smithi193 ceph-mon[21944]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-08-20T06:29:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:06 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T06:29:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:06 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3928543514' entity='client.admin' 2024-08-20T06:29:07.439 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- 172.21.15.107:0/3141508025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 msgr2=0x7f6a54109ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- 172.21.15.107:0/3141508025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54109ad0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009cc0 tx=0x7f6a4c02a8a0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- 172.21.15.107:0/3141508025 shutdown_connections 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- 172.21.15.107:0/3141508025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54109ad0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- 172.21.15.107:0/3141508025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54102790 0x7f6a54102c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- 172.21.15.107:0/3141508025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54101630 0x7f6a54101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- 172.21.15.107:0/3141508025 >> 172.21.15.107:0/3141508025 conn(0x7f6a540fcf00 msgr2=0x7f6a540ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- 172.21.15.107:0/3141508025 shutdown_connections 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- 172.21.15.107:0/3141508025 wait complete. 2024-08-20T06:29:07.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 Processor -- start 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- start start 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54101630 0x7f6a541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102790 0x7f6a54199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5410d070 con 0x7f6a54103150 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5410cef0 con 0x7f6a54102790 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.428+0000 7f6a5adef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5410d1f0 con 0x7f6a54101630 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42860/0 (socket says 172.21.15.107:42860) 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 -- 172.21.15.107:0/1709184993 learned_addr learned my addr 172.21.15.107:0/1709184993 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a58b8b640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54101630 0x7f6a541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a53fff640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102790 0x7f6a54199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:07.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 -- 172.21.15.107:0/1709184993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54101630 msgr2=0x7f6a541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54101630 0x7f6a541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 -- 172.21.15.107:0/1709184993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102790 msgr2=0x7f6a54199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102790 0x7f6a54199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 -- 172.21.15.107:0/1709184993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5419a4f0 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5938c640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c030790 tx=0x7f6a4c0307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004c90 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f6a4c004e30 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5419a780 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c005600 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.429+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a541a0ec0 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.430+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a54101a30 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.431+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f6a4c0057a0 con 0x7f6a54103150 2024-08-20T06:29:07.442 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.433+0000 7f6a51ffb640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 0x7f6a440b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:07.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.433+0000 7f6a58b8b640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 0x7f6a440b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:07.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.433+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6a4c102b20 con 0x7f6a54103150 2024-08-20T06:29:07.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.433+0000 7f6a58b8b640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 0x7f6a440b6080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800abe0 tx=0x7f6a4800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:07.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.437+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6a4c0cbe40 con 0x7f6a54103150 2024-08-20T06:29:07.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.621+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f6a541a1090 con 0x7f6a54103150 2024-08-20T06:29:07.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.627+0000 7f6a51ffb640 1 -- 172.21.15.107:0/1709184993 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v20)=0 v20) ==== 119+0+0 (secure 0 0 0) 0x7f6a4c0cb740 con 0x7f6a54103150 2024-08-20T06:29:07.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.629+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 msgr2=0x7f6a440b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.629+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 0x7f6a440b6080 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800abe0 tx=0x7f6a4800a2d0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.629+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 msgr2=0x7f6a54199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.629+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c030790 tx=0x7f6a4c0307c0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 shutdown_connections 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a54103150 0x7f6a54199c50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a54102790 0x7f6a54199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a440b3bc0 0x7f6a440b6080 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 --2- 172.21.15.107:0/1709184993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a54101630 0x7f6a541989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 >> 172.21.15.107:0/1709184993 conn(0x7f6a540fcf00 msgr2=0x7f6a540fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 shutdown_connections 2024-08-20T06:29:07.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.630+0000 7f6a5adef640 1 -- 172.21.15.107:0/1709184993 wait complete. 2024-08-20T06:29:08.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 -- 172.21.15.107:0/2972244170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 msgr2=0x7f73c810abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:08.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/2972244170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c810abd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f73c0009cd0 tx=0x7f73c002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 -- 172.21.15.107:0/2972244170 shutdown_connections 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/2972244170 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c810abd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/2972244170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 0x7f73c8103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/2972244170 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 0x7f73c8102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 -- 172.21.15.107:0/2972244170 >> 172.21.15.107:0/2972244170 conn(0x7f73c80fe020 msgr2=0x7f73c8100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 -- 172.21.15.107:0/2972244170 shutdown_connections 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.613+0000 7f73cd3c3640 1 -- 172.21.15.107:0/2972244170 wait complete. 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 Processor -- start 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 -- start start 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 0x7f73c8198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 0x7f73c8198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:08.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810e100 con 0x7f73c8104250 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810df80 con 0x7f73c8102730 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73cd3c3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73c810e280 con 0x7f73c8103890 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73c77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73c77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42888/0 (socket says 172.21.15.107:42888) 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73c77fe640 1 -- 172.21.15.107:0/962809576 learned_addr learned my addr 172.21.15.107:0/962809576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73c67fc640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 0x7f73c8198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.614+0000 7f73c6ffd640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 0x7f73c8198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 -- 172.21.15.107:0/962809576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 msgr2=0x7f73c8198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 0x7f73c8198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 -- 172.21.15.107:0/962809576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 msgr2=0x7f73c8198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 0x7f73c8198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 -- 172.21.15.107:0/962809576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c819a380 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73c77fe640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73c00097b0 tx=0x7f73c0003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c0004450 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f73c00045f0 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c819a610 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c003b660 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.615+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c81a0d50 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.616+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c8102b30 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.618+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f73c00305a0 con 0x7f73c8104250 2024-08-20T06:29:08.038 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.619+0000 7f73affff640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 0x7f73b00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.619+0000 7f73c6ffd640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 0x7f73b00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.619+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f73c002e030 con 0x7f73c8104250 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.620+0000 7f73c6ffd640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 0x7f73b00b6000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f73b800abe0 tx=0x7f73b800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.623+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f73c00047a0 con 0x7f73c8104250 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.804+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f73c8063a80 con 0x7f73c8104250 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.810+0000 7f73affff640 1 -- 172.21.15.107:0/962809576 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v21)=0 v21) ==== 122+0+0 (secure 0 0 0) 0x7f73c00098b0 con 0x7f73c8104250 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 msgr2=0x7f73b00b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 0x7f73b00b6000 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f73b800abe0 tx=0x7f73b800a2d0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 msgr2=0x7f73c8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f73c00097b0 tx=0x7f73c0003ef0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.039 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 shutdown_connections 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73c8104250 0x7f73c8199ae0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73b00b3b40 0x7f73b00b6000 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73c8103890 0x7f73c8198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 --2- 172.21.15.107:0/962809576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73c8102730 0x7f73c8198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 >> 172.21.15.107:0/962809576 conn(0x7f73c80fe020 msgr2=0x7f73c8106c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 shutdown_connections 2024-08-20T06:29:08.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:07.813+0000 7f73cd3c3640 1 -- 172.21.15.107:0/962809576 wait complete. 2024-08-20T06:29:08.844 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:08 smithi107 ceph-mon[18495]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:08.844 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:08 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1709184993' entity='client.admin' 2024-08-20T06:29:08.844 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:08 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/962809576' entity='client.admin' 2024-08-20T06:29:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:08 smithi183 ceph-mon[22204]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:08 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1709184993' entity='client.admin' 2024-08-20T06:29:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:08 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/962809576' entity='client.admin' 2024-08-20T06:29:09.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:08 smithi193 ceph-mon[21944]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:09.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:08 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1709184993' entity='client.admin' 2024-08-20T06:29:09.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:08 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/962809576' entity='client.admin' 2024-08-20T06:29:09.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- 172.21.15.107:0/1704276698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8103130 msgr2=0x7f98b8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- 172.21.15.107:0/1704276698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8103130 0x7f98b8109ab0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009cd0 tx=0x7f98b002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- 172.21.15.107:0/1704276698 shutdown_connections 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- 172.21.15.107:0/1704276698 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8103130 0x7f98b8109ab0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- 172.21.15.107:0/1704276698 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8102770 0x7f98b8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- 172.21.15.107:0/1704276698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 0x7f98b8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- 172.21.15.107:0/1704276698 >> 172.21.15.107:0/1704276698 conn(0x7f98b80fcee0 msgr2=0x7f98b80ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- 172.21.15.107:0/1704276698 shutdown_connections 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- 172.21.15.107:0/1704276698 wait complete. 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 Processor -- start 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- start start 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 0x7f98b81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103130 0x7f98b8199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810cfe0 con 0x7f98b8102770 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810ce60 con 0x7f98b8103130 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.618+0000 7f98bfa38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98b810d160 con 0x7f98b8101610 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42934/0 (socket says 172.21.15.107:42934) 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 -- 172.21.15.107:0/2603910014 learned_addr learned my addr 172.21.15.107:0/2603910014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bd7d4640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 0x7f98b81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bdfd5640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103130 0x7f98b8199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 -- 172.21.15.107:0/2603910014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 msgr2=0x7f98b81988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 0x7f98b81988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 -- 172.21.15.107:0/2603910014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103130 msgr2=0x7f98b8199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103130 0x7f98b8199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 -- 172.21.15.107:0/2603910014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b819a400 con 0x7f98b8102770 2024-08-20T06:29:09.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bcfd3640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98a800d950 tx=0x7f98a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8017030 con 0x7f98b8102770 2024-08-20T06:29:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b819a6f0 con 0x7f98b8102770 2024-08-20T06:29:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f98a8004020 con 0x7f98b8102770 2024-08-20T06:29:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8005570 con 0x7f98b8102770 2024-08-20T06:29:09.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.619+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b81a0dd0 con 0x7f98b8102770 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.620+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b81a1000 con 0x7f98b8102770 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.622+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f98a8009a30 con 0x7f98b8102770 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.622+0000 7f98ae7fc640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 0x7f98a40b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.623+0000 7f98bd7d4640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 0x7f98a40b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.623+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98a80e1f40 con 0x7f98b8102770 2024-08-20T06:29:09.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.623+0000 7f98bd7d4640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 0x7f98a40b6000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98b400bbe0 tx=0x7f98b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:09.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.625+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f98a80ab260 con 0x7f98b8102770 2024-08-20T06:29:09.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.807+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f98b81a1790 con 0x7f98b8102770 2024-08-20T06:29:09.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.810+0000 7f98ae7fc640 1 -- 172.21.15.107:0/2603910014 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v24)=0 v24) ==== 127+0+0 (secure 0 0 0) 0x7f98a80aab60 con 0x7f98b8102770 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 msgr2=0x7f98a40b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 0x7f98a40b6000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98b400bbe0 tx=0x7f98b400b390 comp rx=0 tx=0).stop 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 msgr2=0x7f98b8198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98a800d950 tx=0x7f98a800de10 comp rx=0 tx=0).stop 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 shutdown_connections 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98b8103130 0x7f98b8199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98b8102770 0x7f98b8198f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98a40b3b40 0x7f98a40b6000 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 --2- 172.21.15.107:0/2603910014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98b8101610 0x7f98b81988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 >> 172.21.15.107:0/2603910014 conn(0x7f98b80fcee0 msgr2=0x7f98b80fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:09.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 shutdown_connections 2024-08-20T06:29:09.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7f98bfa38640 1 -- 172.21.15.107:0/2603910014 wait complete. 2024-08-20T06:29:09.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 -- 172.21.15.107:0/2534043151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 msgr2=0x7fb484101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 --2- 172.21.15.107:0/2534043151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb484101a10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb478009930 tx=0x7fb47802a420 comp rx=0 tx=0).stop 2024-08-20T06:29:09.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 -- 172.21.15.107:0/2534043151 shutdown_connections 2024-08-20T06:29:09.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 --2- 172.21.15.107:0/2534043151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484103130 0x7fb484109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 --2- 172.21.15.107:0/2534043151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484102770 0x7fb484102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 --2- 172.21.15.107:0/2534043151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb484101a10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 -- 172.21.15.107:0/2534043151 >> 172.21.15.107:0/2534043151 conn(0x7fb4840fcee0 msgr2=0x7fb4840ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 -- 172.21.15.107:0/2534043151 shutdown_connections 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 -- 172.21.15.107:0/2534043151 wait complete. 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.813+0000 7fb48a017640 1 Processor -- start 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 -- start start 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 0x7fb484103c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484103130 0x7fb484104180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb48410d050 con 0x7fb484101610 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb48410ced0 con 0x7fb484103130 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb48a017640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb48410d1d0 con 0x7fb484102770 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42942/0 (socket says 172.21.15.107:42942) 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 -- 172.21.15.107:0/645654432 learned_addr learned my addr 172.21.15.107:0/645654432 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb482ffd640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 0x7fb484103c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb483fff640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484103130 0x7fb484104180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 -- 172.21.15.107:0/645654432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 msgr2=0x7fb484103c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 0x7fb484103c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 -- 172.21.15.107:0/645654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484103130 msgr2=0x7fb484104180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484103130 0x7fb484104180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb4837fe640 1 -- 172.21.15.107:0/645654432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb484104a40 con 0x7fb484101610 2024-08-20T06:29:09.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.814+0000 7fb482ffd640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 0x7fb484103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb4837fe640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb478005ab0 tx=0x7fb478005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47803f070 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fb478003c60 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb478035030 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4841a29d0 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.815+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4841a2e30 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.816+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4841a3060 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.817+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fb478042070 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.818+0000 7fb480ff9640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 0x7fb46c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.818+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb478102d30 con 0x7fb484101610 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.818+0000 7fb482ffd640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 0x7fb46c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:09.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.819+0000 7fb482ffd640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 0x7fb46c0b6000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb474005d00 tx=0x7fb47400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:09.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.824+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb4780cba00 con 0x7fb484101610 2024-08-20T06:29:10.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.005+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fb4841a3730 con 0x7fb484101610 2024-08-20T06:29:10.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.009+0000 7fb480ff9640 1 -- 172.21.15.107:0/645654432 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v25)=0 v25) ==== 120+0+0 (secure 0 0 0) 0x7fb4780cff50 con 0x7fb484101610 2024-08-20T06:29:10.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 msgr2=0x7fb46c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 0x7fb46c0b6000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb474005d00 tx=0x7fb47400b410 comp rx=0 tx=0).stop 2024-08-20T06:29:10.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 msgr2=0x7fb4841053d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb478005ab0 tx=0x7fb478005280 comp rx=0 tx=0).stop 2024-08-20T06:29:10.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 shutdown_connections 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb484103130 0x7fb484104180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb46c0b3b40 0x7fb46c0b6000 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb484102770 0x7fb484103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 --2- 172.21.15.107:0/645654432 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb484101610 0x7fb4841053d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 >> 172.21.15.107:0/645654432 conn(0x7fb4840fcee0 msgr2=0x7fb4840fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 shutdown_connections 2024-08-20T06:29:10.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.011+0000 7fb48a017640 1 -- 172.21.15.107:0/645654432 wait complete. 2024-08-20T06:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:10 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2448945307' entity='client.admin' 2024-08-20T06:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:10 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3013961797' entity='client.admin' 2024-08-20T06:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:10 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2603910014' entity='client.admin' 2024-08-20T06:29:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:10 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/645654432' entity='client.admin' 2024-08-20T06:29:10.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2448945307' entity='client.admin' 2024-08-20T06:29:10.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3013961797' entity='client.admin' 2024-08-20T06:29:10.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2603910014' entity='client.admin' 2024-08-20T06:29:10.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/645654432' entity='client.admin' 2024-08-20T06:29:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:10 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2448945307' entity='client.admin' 2024-08-20T06:29:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:10 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3013961797' entity='client.admin' 2024-08-20T06:29:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:10 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2603910014' entity='client.admin' 2024-08-20T06:29:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:10 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/645654432' entity='client.admin' 2024-08-20T06:29:10.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 -- 172.21.15.107:0/1915326577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd501027a0 msgr2=0x7fdd50102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1915326577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd501027a0 0x7fdd50102c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd38009930 tx=0x7fdd3802a420 comp rx=0 tx=0).stop 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 -- 172.21.15.107:0/1915326577 shutdown_connections 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1915326577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50103160 0x7fdd50109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1915326577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd501027a0 0x7fdd50102c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1915326577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd50101640 0x7fdd50101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 -- 172.21.15.107:0/1915326577 >> 172.21.15.107:0/1915326577 conn(0x7fdd500fcf30 msgr2=0x7fdd500ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 -- 172.21.15.107:0/1915326577 shutdown_connections 2024-08-20T06:29:10.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.616+0000 7fdd548f5640 1 -- 172.21.15.107:0/1915326577 wait complete. 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 Processor -- start 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 -- start start 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50101640 0x7fdd501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd501027a0 0x7fdd50199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5010d010 con 0x7fdd50103160 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5010ce90 con 0x7fdd50101640 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd548f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5010d190 con 0x7fdd501027a0 2024-08-20T06:29:10.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd4ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd4ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42952/0 (socket says 172.21.15.107:42952) 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd4ed9d640 1 -- 172.21.15.107:0/1694715036 learned_addr learned my addr 172.21.15.107:0/1694715036 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd4e59c640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50101640 0x7fdd501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.617+0000 7fdd47fff640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd501027a0 0x7fdd50199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 -- 172.21.15.107:0/1694715036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd501027a0 msgr2=0x7fdd50199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd501027a0 0x7fdd50199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 -- 172.21.15.107:0/1694715036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50101640 msgr2=0x7fdd501989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50101640 0x7fdd501989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 -- 172.21.15.107:0/1694715036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd5019a4f0 con 0x7fdd50103160 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd4ed9d640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800eaa0 tx=0x7fdd4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:10.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800cdb0 con 0x7fdd50103160 2024-08-20T06:29:10.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdd48005020 con 0x7fdd50103160 2024-08-20T06:29:10.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd5019a7e0 con 0x7fdd50103160 2024-08-20T06:29:10.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd480116d0 con 0x7fdd50103160 2024-08-20T06:29:10.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.618+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd501a0ec0 con 0x7fdd50103160 2024-08-20T06:29:10.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.619+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5019aa40 con 0x7fdd50103160 2024-08-20T06:29:10.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.621+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fdd48004a40 con 0x7fdd50103160 2024-08-20T06:29:10.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.622+0000 7fdd477fe640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 0x7fdd3c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:10.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.622+0000 7fdd4e59c640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 0x7fdd3c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:10.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.622+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdd480e2070 con 0x7fdd50103160 2024-08-20T06:29:10.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.622+0000 7fdd4e59c640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 0x7fdd3c0b6000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd400058c0 tx=0x7fdd4000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:10.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.625+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdd48016030 con 0x7fdd50103160 2024-08-20T06:29:10.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.806+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdd501a1090 con 0x7fdd50103160 2024-08-20T06:29:10.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.809+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v26)=0 v26) ==== 123+0+0 (secure 0 0 0) 0x7fdd480aad40 con 0x7fdd50103160 2024-08-20T06:29:10.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.810+0000 7fdd477fe640 1 -- 172.21.15.107:0/1694715036 <== mon.0 v2:172.21.15.107:3300/0 8 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd48005a20 con 0x7fdd50103160 2024-08-20T06:29:10.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 msgr2=0x7fdd3c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 0x7fdd3c0b6000 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd400058c0 tx=0x7fdd4000a2d0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 msgr2=0x7fdd50199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:10.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd4800eaa0 tx=0x7fdd4800ef60 comp rx=0 tx=0).stop 2024-08-20T06:29:10.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 shutdown_connections 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd50103160 0x7fdd50199c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd3c0b3b40 0x7fdd3c0b6000 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd501027a0 0x7fdd50199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 --2- 172.21.15.107:0/1694715036 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd50101640 0x7fdd501989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 >> 172.21.15.107:0/1694715036 conn(0x7fdd500fcf30 msgr2=0x7fdd5006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 shutdown_connections 2024-08-20T06:29:10.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:10.812+0000 7fdd548f5640 1 -- 172.21.15.107:0/1694715036 wait complete. 2024-08-20T06:29:11.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- 172.21.15.107:0/1379758361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 msgr2=0x7fdcd8101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 --2- 172.21.15.107:0/1379758361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8101a50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009930 tx=0x7fdcc802a420 comp rx=0 tx=0).stop 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- 172.21.15.107:0/1379758361 shutdown_connections 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 --2- 172.21.15.107:0/1379758361 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd8103170 0x7fdcd8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 --2- 172.21.15.107:0/1379758361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcd81027b0 0x7fdcd8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 --2- 172.21.15.107:0/1379758361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8101a50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- 172.21.15.107:0/1379758361 >> 172.21.15.107:0/1379758361 conn(0x7fdcd80fcf00 msgr2=0x7fdcd80ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- 172.21.15.107:0/1379758361 shutdown_connections 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- 172.21.15.107:0/1379758361 wait complete. 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 Processor -- start 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 -- start start 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.124+0000 7fdce0249640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 0x7fdcd8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcd8103170 0x7fdcd8199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd810d0a0 con 0x7fdcd8101650 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd810cf20 con 0x7fdcd81027b0 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd810d220 con 0x7fdcd8103170 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42926/0 (socket says 172.21.15.107:42926) 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 -- 172.21.15.107:0/3013961797 learned_addr learned my addr 172.21.15.107:0/3013961797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcdd7e4640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 0x7fdcd8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 -- 172.21.15.107:0/3013961797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcd8103170 msgr2=0x7fdcd8199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcd8103170 0x7fdcd8199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 -- 172.21.15.107:0/3013961797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 msgr2=0x7fdcd8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 0x7fdcd8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 -- 172.21.15.107:0/3013961797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd819a5f0 con 0x7fdcd8101650 2024-08-20T06:29:11.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcdd7e4640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 0x7fdcd8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcddfe5640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009230 tx=0x7fdcc8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc803f030 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7fdcc8032e30 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd819a880 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8035070 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.125+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd81a0e70 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.126+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcd81a10a0 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.127+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fdcc8042030 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.128+0000 7fdcd6ffd640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 0x7fdcc00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.128+0000 7fdcdd7e4640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 0x7fdcc00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.128+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdcc81024c0 con 0x7fdcd8101650 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.129+0000 7fdcdd7e4640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 0x7fdcc00b6000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009930 tx=0x7fdccc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:11.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.132+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdcc80cb7e0 con 0x7fdcd8101650 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.312+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fdcd81a18d0 con 0x7fdcd8101650 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.316+0000 7fdcd6ffd640 1 -- 172.21.15.107:0/3013961797 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v23)=0 v23) ==== 115+0+0 (secure 0 0 0) 0x7fdcc80cb0e0 con 0x7fdcd8101650 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 msgr2=0x7fdcc00b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 0x7fdcc00b6000 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009930 tx=0x7fdccc008040 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 msgr2=0x7fdcd8198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8009230 tx=0x7fdcc8032c10 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 shutdown_connections 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdcc00b3b40 0x7fdcc00b6000 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdcd8103170 0x7fdcd8199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdcd81027b0 0x7fdcd8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 --2- 172.21.15.107:0/3013961797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdcd8101650 0x7fdcd8198ac0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 >> 172.21.15.107:0/3013961797 conn(0x7fdcd80fcf00 msgr2=0x7fdcd806b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 shutdown_connections 2024-08-20T06:29:11.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.318+0000 7fdce0249640 1 -- 172.21.15.107:0/3013961797 wait complete. 2024-08-20T06:29:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:11 smithi193 ceph-mon[21944]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:11 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1694715036' entity='client.admin' 2024-08-20T06:29:11.343 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:11.352 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:11 smithi107 ceph-mon[18495]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:11.352 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:11 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1694715036' entity='client.admin' 2024-08-20T06:29:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:11 smithi183 ceph-mon[22204]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:11 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1694715036' entity='client.admin' 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 -- 172.21.15.107:0/3747326178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 msgr2=0x7fec44102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 --2- 172.21.15.107:0/3747326178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec44102b30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fec38009930 tx=0x7fec3802a420 comp rx=0 tx=0).stop 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 -- 172.21.15.107:0/3747326178 shutdown_connections 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 --2- 172.21.15.107:0/3747326178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44104250 0x7fec4410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 --2- 172.21.15.107:0/3747326178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44103890 0x7fec44103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 --2- 172.21.15.107:0/3747326178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec44102b30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 -- 172.21.15.107:0/3747326178 >> 172.21.15.107:0/3747326178 conn(0x7fec440fe020 msgr2=0x7fec44100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 -- 172.21.15.107:0/3747326178 shutdown_connections 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.924+0000 7fec49de5640 1 -- 172.21.15.107:0/3747326178 wait complete. 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 Processor -- start 2024-08-20T06:29:11.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 -- start start 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 0x7fec44199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44104250 0x7fec44199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410e290 con 0x7fec44102730 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410e110 con 0x7fec44103890 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec49de5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fec4410e410 con 0x7fec44104250 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec437fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42974/0 (socket says 172.21.15.107:42974) 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec437fe640 1 -- 172.21.15.107:0/16937803 learned_addr learned my addr 172.21.15.107:0/16937803 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.925+0000 7fec43fff640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44104250 0x7fec44199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.942 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec42ffd640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 0x7fec44199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 -- 172.21.15.107:0/16937803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44104250 msgr2=0x7fec44199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44104250 0x7fec44199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 -- 172.21.15.107:0/16937803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 msgr2=0x7fec44199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 0x7fec44199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 -- 172.21.15.107:0/16937803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4419a4e0 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec42ffd640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 0x7fec44199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec437fe640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec38009900 tx=0x7fec38003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3803f070 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fec38031d10 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4419a770 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38035030 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.926+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a0ce0 con 0x7fec44102730 2024-08-20T06:29:11.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.927+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec441a0fa0 con 0x7fec44102730 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.928+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fec38042070 con 0x7fec44102730 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.929+0000 7fec40ff9640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 0x7fec2c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.929+0000 7fec42ffd640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 0x7fec2c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.929+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fec38103000 con 0x7fec44102730 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.930+0000 7fec42ffd640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 0x7fec2c0b6000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec3400baa0 tx=0x7fec34005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:11.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:11.932+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fec380cc320 con 0x7fec44102730 2024-08-20T06:29:12.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.120+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fec441a1170 con 0x7fec44102730 2024-08-20T06:29:12.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.124+0000 7fec40ff9640 1 -- 172.21.15.107:0/16937803 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v27)=0 v27) ==== 125+0+0 (secure 0 0 0) 0x7fec380cbc20 con 0x7fec44102730 2024-08-20T06:29:12.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 msgr2=0x7fec2c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:12.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 0x7fec2c0b6000 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fec3400baa0 tx=0x7fec34005d90 comp rx=0 tx=0).stop 2024-08-20T06:29:12.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 msgr2=0x7fec441989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:12.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fec38009900 tx=0x7fec38003d50 comp rx=0 tx=0).stop 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 shutdown_connections 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fec2c0b3b40 0x7fec2c0b6000 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec44104250 0x7fec44199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fec44103890 0x7fec44199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 --2- 172.21.15.107:0/16937803 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec44102730 0x7fec441989b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 >> 172.21.15.107:0/16937803 conn(0x7fec440fe020 msgr2=0x7fec44104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 shutdown_connections 2024-08-20T06:29:12.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:12.127+0000 7fec49de5640 1 -- 172.21.15.107:0/16937803 wait complete. 2024-08-20T06:29:12.357 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi107') 2024-08-20T06:29:12.357 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi107') 2024-08-20T06:29:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:13 smithi107 ceph-mon[18495]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:13 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/16937803' entity='client.admin' 2024-08-20T06:29:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:13 smithi183 ceph-mon[22204]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:13 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/16937803' entity='client.admin' 2024-08-20T06:29:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:13 smithi193 ceph-mon[21944]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:13 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/16937803' entity='client.admin' 2024-08-20T06:29:13.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 -- 172.21.15.107:0/493569959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8103130 msgr2=0x7f98f8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:13.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 --2- 172.21.15.107:0/493569959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8103130 0x7f98f8109ab0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009cd0 tx=0x7f98f002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 -- 172.21.15.107:0/493569959 shutdown_connections 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 --2- 172.21.15.107:0/493569959 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8103130 0x7f98f8109ab0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 --2- 172.21.15.107:0/493569959 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8102770 0x7f98f8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 --2- 172.21.15.107:0/493569959 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8101610 0x7f98f8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 -- 172.21.15.107:0/493569959 >> 172.21.15.107:0/493569959 conn(0x7f98f80fcee0 msgr2=0x7f98f80ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 -- 172.21.15.107:0/493569959 shutdown_connections 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 -- 172.21.15.107:0/493569959 wait complete. 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.843+0000 7f990044d640 1 Processor -- start 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 -- start start 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101610 0x7f98f810be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8103130 0x7f98f810ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:13.765 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810e170 con 0x7f98f8102770 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810dff0 con 0x7f98f8101610 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f990044d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f98f810e2f0 con 0x7f98f8103130 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42908/0 (socket says 172.21.15.107:42908) 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 -- 172.21.15.107:0/2448945307 learned_addr learned my addr 172.21.15.107:0/2448945307 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fe9ea640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8103130 0x7f98f810ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fe1e9640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101610 0x7f98f810be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 -- 172.21.15.107:0/2448945307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8103130 msgr2=0x7f98f810ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8103130 0x7f98f810ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 -- 172.21.15.107:0/2448945307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101610 msgr2=0x7f98f810be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:13.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101610 0x7f98f810be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 -- 172.21.15.107:0/2448945307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f810b500 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98fd9e8640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98e800d950 tx=0x7f98e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.844+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8017070 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.845+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(24 keys) ==== 1032+0+0 (secure 0 0 0) 0x7f98e8009c30 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.845+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e800f620 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.845+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f81a29b0 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.845+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a2ec0 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.846+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f81a30f0 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.848+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f98e8005030 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.849+0000 7f98ef7fe640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 0x7f98e40b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.849+0000 7f98fe1e9640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 0x7f98e40b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.849+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f98e80e1b10 con 0x7f98f8102770 2024-08-20T06:29:13.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.849+0000 7f98fe1e9640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 0x7f98e40b6000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f40058c0 tx=0x7f98f400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:08.851+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f98e80aae30 con 0x7f98f8102770 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.032+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f98f81a32c0 con 0x7f98f8102770 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.035+0000 7f98ef7fe640 1 -- 172.21.15.107:0/2448945307 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f98e8009de0 con 0x7f98f8102770 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 msgr2=0x7f98e40b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 0x7f98e40b6000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98f40058c0 tx=0x7f98f400a4c0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 msgr2=0x7f98f810a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f98e800d950 tx=0x7f98e800de10 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 shutdown_connections 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f98e40b3b40 0x7f98e40b6000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98f8103130 0x7f98f810ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f98f8102770 0x7f98f810a700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 --2- 172.21.15.107:0/2448945307 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f98f8101610 0x7f98f810be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:13.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 >> 172.21.15.107:0/2448945307 conn(0x7f98f80fcee0 msgr2=0x7f98f8103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 shutdown_connections 2024-08-20T06:29:13.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:09.038+0000 7f990044d640 1 -- 172.21.15.107:0/2448945307 wait complete. 2024-08-20T06:29:15.462 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:15 smithi183 ceph-mon[22204]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:15 smithi193 ceph-mon[21944]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:15.566 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:15 smithi107 ceph-mon[18495]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:15.570 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:15.906 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:15.907 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:15.907 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:17.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:17 smithi107 ceph-mon[18495]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:17.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:17 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:17 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:17 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:17 smithi183 ceph-mon[22204]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:17 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:17 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:17 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:17 smithi193 ceph-mon[21944]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:17 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:17 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:17 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:18.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 -- 172.21.15.107:0/1256485666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 msgr2=0x7fa4c8109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/1256485666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c8109ae0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0009cd0 tx=0x7fa4c002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 -- 172.21.15.107:0/1256485666 shutdown_connections 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/1256485666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c8109ae0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/1256485666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c81027a0 0x7fa4c8102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/1256485666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c8101640 0x7fa4c8101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 -- 172.21.15.107:0/1256485666 >> 172.21.15.107:0/1256485666 conn(0x7fa4c80fcf30 msgr2=0x7fa4c80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 -- 172.21.15.107:0/1256485666 shutdown_connections 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.192+0000 7fa4cdab0640 1 -- 172.21.15.107:0/1256485666 wait complete. 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 Processor -- start 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 -- start start 2024-08-20T06:29:18.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c8101640 0x7fa4c806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c81027a0 0x7fa4c819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c810d220 con 0x7fa4c8103160 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c810d0a0 con 0x7fa4c81027a0 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4cdab0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa4c810d3a0 con 0x7fa4c8101640 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42692/0 (socket says 172.21.15.107:42692) 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 -- 172.21.15.107:0/4204212434 learned_addr learned my addr 172.21.15.107:0/4204212434 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c77fe640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c8101640 0x7fa4c806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 -- 172.21.15.107:0/4204212434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c8101640 msgr2=0x7fa4c806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c6ffd640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c81027a0 0x7fa4c819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c8101640 0x7fa4c806bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 -- 172.21.15.107:0/4204212434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c81027a0 msgr2=0x7fa4c819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c81027a0 0x7fa4c819d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.193+0000 7fa4c7fff640 1 -- 172.21.15.107:0/4204212434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4c81a0630 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4c7fff640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0031040 tx=0x7fa4c0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c0004240 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4c81a08c0 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa4c00305a0 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4c003a660 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.194+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4c81a0e50 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.195+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4c8108960 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.196+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fa4c00309e0 con 0x7fa4c8103160 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.197+0000 7fa4c4ff9640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 0x7fa4b00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.197+0000 7fa4c77fe640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 0x7fa4b00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.197+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa4c002e030 con 0x7fa4c8103160 2024-08-20T06:29:18.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.198+0000 7fa4c77fe640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 0x7fa4b00b6000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800abe0 tx=0x7fa4b800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.200+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa4c00cca50 con 0x7fa4c8103160 2024-08-20T06:29:18.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.381+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fa4c802e340 con 0x7fa4c8103160 2024-08-20T06:29:18.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.387+0000 7fa4c4ff9640 1 -- 172.21.15.107:0/4204212434 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v28)=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7fa4c00cc350 con 0x7fa4c8103160 2024-08-20T06:29:18.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 msgr2=0x7fa4b00b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 0x7fa4b00b6000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4b800abe0 tx=0x7fa4b800a2d0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 msgr2=0x7fa4c819fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa4c0031040 tx=0x7fa4c0003c10 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 shutdown_connections 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4c8103160 0x7fa4c819fd70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa4c81027a0 0x7fa4c819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa4b00b3b40 0x7fa4b00b6000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 --2- 172.21.15.107:0/4204212434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4c8101640 0x7fa4c806bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 >> 172.21.15.107:0/4204212434 conn(0x7fa4c80fcf30 msgr2=0x7fa4c8105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 shutdown_connections 2024-08-20T06:29:18.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.390+0000 7fa4cdab0640 1 -- 172.21.15.107:0/4204212434 wait complete. 2024-08-20T06:29:18.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 -- 172.21.15.107:0/3128414019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8103130 msgr2=0x7f0da8109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3128414019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8103130 0x7f0da8109ab0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0da0009cd0 tx=0x7f0da002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 -- 172.21.15.107:0/3128414019 shutdown_connections 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3128414019 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8103130 0x7f0da8109ab0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3128414019 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3128414019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8101610 0x7f0da8101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 -- 172.21.15.107:0/3128414019 >> 172.21.15.107:0/3128414019 conn(0x7f0da80fcee0 msgr2=0x7f0da80ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 -- 172.21.15.107:0/3128414019 shutdown_connections 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.398+0000 7f0db03f6640 1 -- 172.21.15.107:0/3128414019 wait complete. 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 Processor -- start 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 -- start start 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8103130 0x7f0da8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810d120 con 0x7f0da8101610 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810cfa0 con 0x7f0da8102770 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0db03f6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0da810d2a0 con 0x7f0da8103130 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42702/0 (socket says 172.21.15.107:42702) 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 -- 172.21.15.107:0/3117875613 learned_addr learned my addr 172.21.15.107:0/3117875613 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae993640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8103130 0x7f0da8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dad991640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 -- 172.21.15.107:0/3117875613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8103130 msgr2=0x7f0da8199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8103130 0x7f0da8199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 -- 172.21.15.107:0/3117875613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 msgr2=0x7f0da8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dae192640 1 -- 172.21.15.107:0/3117875613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da819a4c0 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.399+0000 7f0dad991640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0dae192640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0da400d950 tx=0x7f0da400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da4017070 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0da4009c30 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da819a7b0 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da400f620 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.400+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a0e90 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.401+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da81a1060 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.402+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f0da4010460 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.403+0000 7f0d9f7fe640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 0x7f0d940b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.403+0000 7f0dad991640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 0x7f0d940b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.403+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0da40e1b60 con 0x7f0da8101610 2024-08-20T06:29:18.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.403+0000 7f0dad991640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 0x7f0d940b6000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009b40 tx=0x7f0d98005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.407+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0da40aae80 con 0x7f0da8101610 2024-08-20T06:29:18.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.585+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f0da81a1230 con 0x7f0da8101610 2024-08-20T06:29:18.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.591+0000 7f0d9f7fe640 1 -- 172.21.15.107:0/3117875613 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f0da4009de0 con 0x7f0da8101610 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 msgr2=0x7f0d940b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 0x7f0d940b6000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009b40 tx=0x7f0d98005e50 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 msgr2=0x7f0da8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0da400d950 tx=0x7f0da400de10 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 shutdown_connections 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0d940b3b40 0x7f0d940b6000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da8103130 0x7f0da8199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0da8102770 0x7f0da8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 --2- 172.21.15.107:0/3117875613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0da8101610 0x7f0da8198990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 >> 172.21.15.107:0/3117875613 conn(0x7f0da80fcee0 msgr2=0x7f0da80fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 shutdown_connections 2024-08-20T06:29:18.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.593+0000 7f0db03f6640 1 -- 172.21.15.107:0/3117875613 wait complete. 2024-08-20T06:29:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 -- 172.21.15.107:0/1120408819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 msgr2=0x7f931010abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 --2- 172.21.15.107:0/1120408819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f931010abd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f9308009cd0 tx=0x7f930802a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 -- 172.21.15.107:0/1120408819 shutdown_connections 2024-08-20T06:29:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 --2- 172.21.15.107:0/1120408819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f931010abd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 --2- 172.21.15.107:0/1120408819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310103890 0x7f9310103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 --2- 172.21.15.107:0/1120408819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310102730 0x7f9310102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 -- 172.21.15.107:0/1120408819 >> 172.21.15.107:0/1120408819 conn(0x7f93100fe020 msgr2=0x7f9310100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 -- 172.21.15.107:0/1120408819 shutdown_connections 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 -- 172.21.15.107:0/1120408819 wait complete. 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.769+0000 7f9316463640 1 Processor -- start 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 -- start start 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310102730 0x7f93101989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310103890 0x7f9310199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e290 con 0x7f9310104250 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e110 con 0x7f9310102730 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9316463640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f931010e410 con 0x7f9310103890 2024-08-20T06:29:18.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42712/0 (socket says 172.21.15.107:42712) 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 -- 172.21.15.107:0/2588419290 learned_addr learned my addr 172.21.15.107:0/2588419290 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f930ffff640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310102730 0x7f93101989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f930f7fe640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310103890 0x7f9310199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 -- 172.21.15.107:0/2588419290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310103890 msgr2=0x7f9310199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310103890 0x7f9310199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 -- 172.21.15.107:0/2588419290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310102730 msgr2=0x7f93101989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310102730 0x7f93101989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 -- 172.21.15.107:0/2588419290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931019a4f0 con 0x7f9310104250 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.770+0000 7f9314a00640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9308031040 tx=0x7f9308003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.771+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308004240 con 0x7f9310104250 2024-08-20T06:29:18.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.771+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93080305a0 con 0x7f9310104250 2024-08-20T06:29:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.771+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931019a780 con 0x7f9310104250 2024-08-20T06:29:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.771+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080466e0 con 0x7f9310104250 2024-08-20T06:29:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.771+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a0ec0 con 0x7f9310104250 2024-08-20T06:29:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.772+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9310102b30 con 0x7f9310104250 2024-08-20T06:29:18.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.773+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f9308030a10 con 0x7f9310104250 2024-08-20T06:29:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.774+0000 7f930d7fa640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 0x7f92f80b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.774+0000 7f930ffff640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 0x7f92f80b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.774+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f930802e030 con 0x7f9310104250 2024-08-20T06:29:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.774+0000 7f930ffff640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 0x7f92f80b6080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93000058c0 tx=0x7f930000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:18.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.777+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f930804b060 con 0x7f9310104250 2024-08-20T06:29:18.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.954+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f9310109a50 con 0x7f9310104250 2024-08-20T06:29:18.960 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.960+0000 7f930d7fa640 1 -- 172.21.15.107:0/2588419290 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v30)=0 v30) ==== 121+0+0 (secure 0 0 0) 0x7f9310109a50 con 0x7f9310104250 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 msgr2=0x7f92f80b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 0x7f92f80b6080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f93000058c0 tx=0x7f930000a2d0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 msgr2=0x7f9310199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9308031040 tx=0x7f9308003ba0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 shutdown_connections 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9310104250 0x7f9310199c50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f92f80b3bc0 0x7f92f80b6080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9310103890 0x7f9310199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 --2- 172.21.15.107:0/2588419290 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9310102730 0x7f93101989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:18.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 >> 172.21.15.107:0/2588419290 conn(0x7f93100fe020 msgr2=0x7f9310106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:18.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 shutdown_connections 2024-08-20T06:29:18.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:18.962+0000 7f9316463640 1 -- 172.21.15.107:0/2588419290 wait complete. 2024-08-20T06:29:19.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:19 smithi107 ceph-mon[18495]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:19 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4204212434' entity='client.admin' 2024-08-20T06:29:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:19 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3117875613' entity='client.admin' 2024-08-20T06:29:19.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:19 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2588419290' entity='client.admin' 2024-08-20T06:29:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:19 smithi183 ceph-mon[22204]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:19 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4204212434' entity='client.admin' 2024-08-20T06:29:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:19 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3117875613' entity='client.admin' 2024-08-20T06:29:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:19 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2588419290' entity='client.admin' 2024-08-20T06:29:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:19 smithi193 ceph-mon[21944]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:19 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4204212434' entity='client.admin' 2024-08-20T06:29:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:19 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3117875613' entity='client.admin' 2024-08-20T06:29:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:19 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2588419290' entity='client.admin' 2024-08-20T06:29:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 -- 172.21.15.107:0/223345699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c1027b0 msgr2=0x7f013c102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 --2- 172.21.15.107:0/223345699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c1027b0 0x7f013c102c30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f012c009930 tx=0x7f012c02a420 comp rx=0 tx=0).stop 2024-08-20T06:29:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 -- 172.21.15.107:0/223345699 shutdown_connections 2024-08-20T06:29:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 --2- 172.21.15.107:0/223345699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 0x7f013c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 --2- 172.21.15.107:0/223345699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c1027b0 0x7f013c102c30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 --2- 172.21.15.107:0/223345699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c101650 0x7f013c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 -- 172.21.15.107:0/223345699 >> 172.21.15.107:0/223345699 conn(0x7f013c0fcf00 msgr2=0x7f013c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 -- 172.21.15.107:0/223345699 shutdown_connections 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.829+0000 7f0143f9e640 1 -- 172.21.15.107:0/223345699 wait complete. 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 Processor -- start 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 -- start start 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c1027b0 0x7f013c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 0x7f013c199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:19.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10d180 con 0x7f013c101650 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10d000 con 0x7f013c1027b0 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0143f9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10d300 con 0x7f013c103170 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0141d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0141d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:42734/0 (socket says 172.21.15.107:42734) 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0141d3a640 1 -- 172.21.15.107:0/639341731 learned_addr learned my addr 172.21.15.107:0/639341731 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f0141539640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c1027b0 0x7f013c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.830+0000 7f014253b640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 0x7f013c199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 -- 172.21.15.107:0/639341731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 msgr2=0x7f013c199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 0x7f013c199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 -- 172.21.15.107:0/639341731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c1027b0 msgr2=0x7f013c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c1027b0 0x7f013c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 -- 172.21.15.107:0/639341731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c19a600 con 0x7f013c101650 2024-08-20T06:29:19.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0141d3a640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f013800d950 tx=0x7f013800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138017070 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0138009c30 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013800f620 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c19a8f0 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.831+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c1a0fd0 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.832+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c1a11a0 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.833+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f0138010460 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.834+0000 7f0132ffd640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 0x7f01280b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.834+0000 7f0141539640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 0x7f01280b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.835+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f01380e1ad0 con 0x7f013c101650 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.835+0000 7f0141539640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 0x7f01280b6000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:19.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:19.838+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0138009de0 con 0x7f013c101650 2024-08-20T06:29:20.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.019+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f013c1a1370 con 0x7f013c101650 2024-08-20T06:29:20.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.022+0000 7f0132ffd640 1 -- 172.21.15.107:0/639341731 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v31)=0 v31) ==== 116+0+0 (secure 0 0 0) 0x7f01380aa780 con 0x7f013c101650 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 msgr2=0x7f01280b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 0x7f01280b6000 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f012c009900 tx=0x7f012c005720 comp rx=0 tx=0).stop 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 msgr2=0x7f013c198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f013800d950 tx=0x7f013800de10 comp rx=0 tx=0).stop 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 shutdown_connections 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f01280b3b40 0x7f01280b6000 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f013c103170 0x7f013c199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:20.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f013c1027b0 0x7f013c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:20.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 --2- 172.21.15.107:0/639341731 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f013c101650 0x7f013c198ad0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:20.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 >> 172.21.15.107:0/639341731 conn(0x7f013c0fcf00 msgr2=0x7f013c06b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:20.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 shutdown_connections 2024-08-20T06:29:20.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:20.025+0000 7f0143f9e640 1 -- 172.21.15.107:0/639341731 wait complete. 2024-08-20T06:29:20.336 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi107') 2024-08-20T06:29:20.336 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi107') 2024-08-20T06:29:20.336 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi107') 2024-08-20T06:29:20.336 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi107') 2024-08-20T06:29:20.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:20.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:20.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:20.410 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:20 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/639341731' entity='client.admin' 2024-08-20T06:29:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:20 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/639341731' entity='client.admin' 2024-08-20T06:29:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:20.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:20 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/639341731' entity='client.admin' 2024-08-20T06:29:21.242 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:21 smithi107 ceph-mon[18495]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.432 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.433 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', '1c223f02-5ebd-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi107') 2024-08-20T06:29:21.433 INFO:tasks.cephadm:Deploying OSDs... 2024-08-20T06:29:21.433 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:29:21.433 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T06:29:21.450 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T06:29:21.450 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-08-20T06:29:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:21 smithi183 ceph-mon[22204]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:21.506 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T06:29:21.506 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-20T06:29:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:21.506 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-08-20 06:28:12.375230250 +0000 2024-08-20T06:29:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-08-20 06:24:04.956510682 +0000 2024-08-20T06:29:21.507 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-08-20 06:24:04.956510682 +0000 2024-08-20T06:29:21.507 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-08-20 06:24:04.956510682 +0000 2024-08-20T06:29:21.507 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T06:29:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:21 smithi193 ceph-mon[21944]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:21.572 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-08-20T06:29:21.573 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-08-20T06:29:21.573 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000310009 s, 1.7 MB/s 2024-08-20T06:29:21.574 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T06:29:21.631 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-08-20 06:28:12.406229732 +0000 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-08-20 06:24:05.290511328 +0000 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-08-20 06:24:05.290511328 +0000 2024-08-20T06:29:21.686 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-08-20 06:24:05.290511328 +0000 2024-08-20T06:29:21.687 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T06:29:21.750 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-08-20T06:29:21.750 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-08-20T06:29:21.750 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000190561 s, 2.7 MB/s 2024-08-20T06:29:21.751 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T06:29:21.808 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-08-20T06:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T06:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-08-20T06:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:21.863 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:21.864 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-08-20 06:28:12.436229231 +0000 2024-08-20T06:29:21.864 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-08-20 06:24:05.617511961 +0000 2024-08-20T06:29:21.864 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-08-20 06:24:05.617511961 +0000 2024-08-20T06:29:21.864 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-08-20 06:24:05.617511961 +0000 2024-08-20T06:29:21.864 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T06:29:21.926 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-08-20T06:29:21.926 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-08-20T06:29:21.926 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000176212 s, 2.9 MB/s 2024-08-20T06:29:21.928 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T06:29:21.983 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-08-20 06:28:12.466228731 +0000 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-08-20 06:24:05.955512615 +0000 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-08-20 06:24:05.955512615 +0000 2024-08-20T06:29:22.039 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-08-20 06:24:05.955512615 +0000 2024-08-20T06:29:22.040 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T06:29:22.103 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-08-20T06:29:22.103 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-08-20T06:29:22.103 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000216716 s, 2.4 MB/s 2024-08-20T06:29:22.104 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T06:29:22.161 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-08-20T06:29:22.161 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T06:29:22.181 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T06:29:22.181 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-08-20 06:28:59.785538246 +0000 2024-08-20T06:29:22.239 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-08-20 06:24:05.414008679 +0000 2024-08-20T06:29:22.240 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-08-20 06:24:05.414008679 +0000 2024-08-20T06:29:22.240 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-08-20 06:24:05.414008679 +0000 2024-08-20T06:29:22.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T06:29:22.309 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-08-20T06:29:22.309 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-08-20T06:29:22.309 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000268465 s, 1.9 MB/s 2024-08-20T06:29:22.311 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T06:29:22.369 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-08-20 06:28:59.785538246 +0000 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-08-20 06:24:05.737999503 +0000 2024-08-20T06:29:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-08-20 06:24:05.737999503 +0000 2024-08-20T06:29:22.427 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-08-20 06:24:05.737999503 +0000 2024-08-20T06:29:22.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T06:29:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-08-20T06:29:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-08-20T06:29:22.492 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000205995 s, 2.5 MB/s 2024-08-20T06:29:22.493 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T06:29:22.551 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-08-20 06:28:59.785538246 +0000 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-08-20 06:24:06.106989053 +0000 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-08-20 06:24:06.106989053 +0000 2024-08-20T06:29:22.608 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-08-20 06:24:06.106989053 +0000 2024-08-20T06:29:22.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T06:29:22.672 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-08-20T06:29:22.672 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-08-20T06:29:22.672 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000213657 s, 2.4 MB/s 2024-08-20T06:29:22.673 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T06:29:22.730 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-08-20T06:29:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T06:29:22.786 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.786 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-08-20T06:29:22.786 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.787 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.787 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-08-20 06:28:59.786538218 +0000 2024-08-20T06:29:22.787 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-08-20 06:24:06.441979565 +0000 2024-08-20T06:29:22.787 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-08-20 06:24:06.441979565 +0000 2024-08-20T06:29:22.787 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-08-20 06:24:06.441979565 +0000 2024-08-20T06:29:22.787 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T06:29:22.851 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-08-20T06:29:22.851 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-08-20T06:29:22.851 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000219235 s, 2.3 MB/s 2024-08-20T06:29:22.853 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T06:29:22.910 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-08-20T06:29:22.910 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T06:29:22.930 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T06:29:22.930 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-08-20T06:29:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T06:29:22.989 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:22.989 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-08-20 06:28:57.361394958 +0000 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-08-20 06:24:05.070276855 +0000 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-08-20 06:24:05.070276855 +0000 2024-08-20T06:29:22.990 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-08-20 06:24:05.070276855 +0000 2024-08-20T06:29:22.990 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T06:29:23.061 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-08-20T06:29:23.061 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-08-20T06:29:23.061 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000221603 s, 2.3 MB/s 2024-08-20T06:29:23.063 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T06:29:23.122 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-08-20T06:29:23.180 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-08-20 06:28:57.362394943 +0000 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-08-20 06:24:05.403277122 +0000 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-08-20 06:24:05.403277122 +0000 2024-08-20T06:29:23.181 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-08-20 06:24:05.403277122 +0000 2024-08-20T06:29:23.181 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T06:29:23.247 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-08-20T06:29:23.247 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-08-20T06:29:23.247 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.0001932 s, 2.7 MB/s 2024-08-20T06:29:23.248 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T06:29:23.306 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-08-20 06:28:57.362394943 +0000 2024-08-20T06:29:23.363 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-08-20 06:24:05.732277387 +0000 2024-08-20T06:29:23.364 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-08-20 06:24:05.732277387 +0000 2024-08-20T06:29:23.364 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-08-20 06:24:05.732277387 +0000 2024-08-20T06:29:23.364 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T06:29:23.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:23 smithi193 ceph-mon[21944]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:23.428 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-08-20T06:29:23.428 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-08-20T06:29:23.428 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000187419 s, 2.7 MB/s 2024-08-20T06:29:23.429 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T06:29:23.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:23 smithi183 ceph-mon[22204]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:23.487 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-08-20 06:28:57.362394943 +0000 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-08-20 06:24:06.055277647 +0000 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-08-20 06:24:06.055277647 +0000 2024-08-20T06:29:23.543 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-08-20 06:24:06.055277647 +0000 2024-08-20T06:29:23.544 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T06:29:23.607 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-08-20T06:29:23.607 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-08-20T06:29:23.607 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000209348 s, 2.4 MB/s 2024-08-20T06:29:23.608 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T06:29:23.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:23 smithi107 ceph-mon[18495]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:23.665 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-08-20T06:29:23.666 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T06:29:23.915 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:25.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:25 smithi183 ceph-mon[22204]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:25.248 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:25 smithi107 ceph-mon[18495]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:25 smithi193 ceph-mon[21944]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:26.306 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:29:26.333 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-08-20T06:29:26.574 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:29:27.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 -- 172.21.15.107:0/4152149319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 msgr2=0x7f3f24103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:27.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/4152149319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24103d10 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009930 tx=0x7f3f1802a420 comp rx=0 tx=0).stop 2024-08-20T06:29:27.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 -- 172.21.15.107:0/4152149319 shutdown_connections 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/4152149319 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 0x7f3f2410abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/4152149319 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24103d10 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/4152149319 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 -- 172.21.15.107:0/4152149319 >> 172.21.15.107:0/4152149319 conn(0x7f3f240fe020 msgr2=0x7f3f24100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.073+0000 7f3f28ca4640 1 -- 172.21.15.107:0/4152149319 shutdown_connections 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 -- 172.21.15.107:0/4152149319 wait complete. 2024-08-20T06:29:27.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 Processor -- start 2024-08-20T06:29:27.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 -- start start 2024-08-20T06:29:27.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:27.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 0x7f3f24199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e260 con 0x7f3f24103890 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e0e0 con 0x7f3f24102730 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.074+0000 7f3f28ca4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2410e3e0 con 0x7f3f24104250 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:49310/0 (socket says 172.21.15.107:49310) 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f2259c640 1 -- 172.21.15.107:0/3593221728 learned_addr learned my addr 172.21.15.107:0/3593221728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f22d9d640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 0x7f3f24199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 -- 172.21.15.107:0/3593221728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 msgr2=0x7f3f24199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:27.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 0x7f3f24199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 -- 172.21.15.107:0/3593221728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 msgr2=0x7f3f24198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 -- 172.21.15.107:0/3593221728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2419a560 con 0x7f3f24103890 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f21d9b640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24199080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005790 tx=0x7f3f180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1803f070 con 0x7f3f24103890 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2419a7f0 con 0x7f3f24103890 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3f1803b460 con 0x7f3f24103890 2024-08-20T06:29:27.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f18035070 con 0x7f3f24103890 2024-08-20T06:29:27.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.075+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f24075280 con 0x7f3f24103890 2024-08-20T06:29:27.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.076+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f24102b30 con 0x7f3f24103890 2024-08-20T06:29:27.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.079+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f3f1803b910 con 0x7f3f24103890 2024-08-20T06:29:27.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.079+0000 7f3f137fe640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 0x7f3f0c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:27.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.079+0000 7f3f2259c640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 0x7f3f0c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:27.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.079+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3f181029d0 con 0x7f3f24103890 2024-08-20T06:29:27.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.080+0000 7f3f2259c640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 0x7f3f0c0b6000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f14005fd0 tx=0x7f3f1400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:27.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.082+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3f180cb6a0 con 0x7f3f24103890 2024-08-20T06:29:27.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:27.270+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3f24105380 con 0x7f3f0c0b3b40 2024-08-20T06:29:27.272 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:27 smithi107 ceph-mon[18495]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:27 smithi183 ceph-mon[22204]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:27 smithi193 ceph-mon[21944]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:28.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:28.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:28.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:28.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:28.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:28.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:29 smithi183 ceph-mon[22204]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:29 smithi183 ceph-mon[22204]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:29 smithi193 ceph-mon[21944]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:29 smithi193 ceph-mon[21944]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:29 smithi107 ceph-mon[18495]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:29 smithi107 ceph-mon[18495]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:30 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"} : dispatch 2024-08-20T06:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:30 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"}]': finished 2024-08-20T06:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:30 smithi183 ceph-mon[22204]: osdmap e5: 1 total, 0 up, 1 in 2024-08-20T06:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:30 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2184972566' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:30 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"} : dispatch 2024-08-20T06:29:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:30 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"}]': finished 2024-08-20T06:29:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:30 smithi193 ceph-mon[21944]: osdmap e5: 1 total, 0 up, 1 in 2024-08-20T06:29:30.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:30.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:30 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2184972566' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"} : dispatch 2024-08-20T06:29:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2515104999' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fb671ab-2dd3-4934-9bf5-4492165e868c"}]': finished 2024-08-20T06:29:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:30 smithi107 ceph-mon[18495]: osdmap e5: 1 total, 0 up, 1 in 2024-08-20T06:29:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2184972566' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:31.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:31 smithi183 ceph-mon[22204]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:31 smithi193 ceph-mon[21944]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:31.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:31 smithi107 ceph-mon[18495]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:33.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:33 smithi183 ceph-mon[22204]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:33 smithi193 ceph-mon[21944]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:33 smithi107 ceph-mon[18495]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:35.340 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:35 smithi107 ceph-mon[18495]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:35 smithi183 ceph-mon[22204]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:35 smithi193 ceph-mon[21944]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:37.285 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:37 smithi107 ceph-mon[18495]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:37 smithi183 ceph-mon[22204]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:37 smithi193 ceph-mon[21944]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:38.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:38 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T06:29:38.428 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:38 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:38 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T06:29:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:38 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:38 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T06:29:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:38 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:39.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:39 smithi107 ceph-mon[18495]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:39.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:39 smithi107 ceph-mon[18495]: Deploying daemon osd.0 on smithi107 2024-08-20T06:29:39.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:39 smithi183 ceph-mon[22204]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:39 smithi183 ceph-mon[22204]: Deploying daemon osd.0 on smithi107 2024-08-20T06:29:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:39 smithi193 ceph-mon[21944]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:39 smithi193 ceph-mon[21944]: Deploying daemon osd.0 on smithi107 2024-08-20T06:29:41.255 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:41 smithi107 ceph-mon[18495]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:41 smithi183 ceph-mon[22204]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:41 smithi193 ceph-mon[21944]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:43.483 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:43 smithi107 ceph-mon[18495]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:43.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:43.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:43 smithi183 ceph-mon[22204]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:43 smithi193 ceph-mon[21944]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.458 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:45.458 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:45.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:45.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:45.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:45.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:45.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:47.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.223+0000 7f3f137fe640 1 -- 172.21.15.107:0/3593221728 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3f24105380 con 0x7f3f0c0b3b40 2024-08-20T06:29:47.224 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-08-20T06:29:47.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 msgr2=0x7f3f0c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:47.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 0x7f3f0c0b6000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3f14005fd0 tx=0x7f3f1400b480 comp rx=0 tx=0).stop 2024-08-20T06:29:47.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 msgr2=0x7f3f24199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:47.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24199080 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3f18005790 tx=0x7f3f180042f0 comp rx=0 tx=0).stop 2024-08-20T06:29:47.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 shutdown_connections 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3f0c0b3b40 0x7f3f0c0b6000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f24104250 0x7f3f24199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3f24103890 0x7f3f24199080 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 --2- 172.21.15.107:0/3593221728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3f24102730 0x7f3f24198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 >> 172.21.15.107:0/3593221728 conn(0x7f3f240fe020 msgr2=0x7f3f24104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 shutdown_connections 2024-08-20T06:29:47.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:29:47.225+0000 7f3f28ca4640 1 -- 172.21.15.107:0/3593221728 wait complete. 2024-08-20T06:29:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:47 smithi183 ceph-mon[22204]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:47.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:47 smithi107 ceph-mon[18495]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:47 smithi193 ceph-mon[21944]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:47.844 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.0.service 2024-08-20T06:29:47.847 INFO:tasks.cephadm:Deploying osd.1 on smithi183 with /dev/vg_nvme/lv_4... 2024-08-20T06:29:47.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T06:29:48.094 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:29:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:48.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:49.260 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:49 smithi107 ceph-mon[18495]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:49 smithi183 ceph-mon[22204]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:49 smithi193 ceph-mon[21944]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:49.611 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:29:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:29:49.373+0000 7f883f571640 -1 osd.0 0 log_to_monitors true 2024-08-20T06:29:50.273 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:50 smithi107 ceph-mon[18495]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T06:29:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:50 smithi183 ceph-mon[22204]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T06:29:50.528 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:29:50.552 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_4 2024-08-20T06:29:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:50 smithi193 ceph-mon[21944]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T06:29:50.797 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:29:51.197 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 -- 172.21.15.183:0/295172608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 msgr2=0x7f7758109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 --2- 172.21.15.183:0/295172608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758109ab0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7750009cd0 tx=0x7f775002a8e0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 -- 172.21.15.183:0/295172608 shutdown_connections 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 --2- 172.21.15.183:0/295172608 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758109ab0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 --2- 172.21.15.183:0/295172608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 0x7f7758102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 --2- 172.21.15.183:0/295172608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f7758101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 -- 172.21.15.183:0/295172608 >> 172.21.15.183:0/295172608 conn(0x7f77580fcee0 msgr2=0x7f77580ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 -- 172.21.15.183:0/295172608 shutdown_connections 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 -- 172.21.15.183:0/295172608 wait complete. 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.196+0000 7f775efd2640 1 Processor -- start 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 -- start start 2024-08-20T06:29:51.198 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 0x7f7758199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f775810d140 con 0x7f7758101610 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f775810cfc0 con 0x7f7758103130 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775efd2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f775810d2c0 con 0x7f7758102770 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775d56f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.183:58480/0 (socket says 172.21.15.183:58480) 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f7757fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 0x7f7758199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:51.199 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 -- 172.21.15.183:0/2131407010 learned_addr learned my addr 172.21.15.183:0/2131407010 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 -- 172.21.15.183:0/2131407010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 msgr2=0x7f7758199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 0x7f7758199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 -- 172.21.15.183:0/2131407010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 msgr2=0x7f7758199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 -- 172.21.15.183:0/2131407010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775819a4f0 con 0x7f7758101610 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775d56f640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.197+0000 7f775cd6e640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e9f0 tx=0x7f774c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.198+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cd80 con 0x7f7758101610 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.198+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775819a7e0 con 0x7f7758101610 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.198+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f774c0108c0 con 0x7f7758101610 2024-08-20T06:29:51.201 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.198+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0187c0 con 0x7f7758101610 2024-08-20T06:29:51.202 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.198+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7758103c50 con 0x7f7758101610 2024-08-20T06:29:51.202 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.199+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7758103eb0 con 0x7f7758101610 2024-08-20T06:29:51.202 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.200+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f774c010430 con 0x7f7758101610 2024-08-20T06:29:51.203 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.201+0000 7f7755ffb640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 0x7f77480b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:29:51.203 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.201+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1361+0+0 (secure 0 0 0) 0x7f774c0e2410 con 0x7f7758101610 2024-08-20T06:29:51.203 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.201+0000 7f7757fff640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 0x7f77480b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:29:51.203 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.202+0000 7f7757fff640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 0x7f77480b6000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7744005fd0 tx=0x7f774400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:29:51.205 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.204+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f774c0aaf70 con 0x7f7758101610 2024-08-20T06:29:51.398 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:29:51.397+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f77580ff0a0 con 0x7f77480b3b40 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:51.399 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:29:51.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:51.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:51.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:29:51.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:29:52.450 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi107 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi107 to 17087M 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:52.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:52 smithi183 ceph-mon[22204]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' 2024-08-20T06:29:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi107 2024-08-20T06:29:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi107 to 17087M 2024-08-20T06:29:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:29:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T06:29:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:52.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:52.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:52.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:52 smithi193 ceph-mon[21944]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' 2024-08-20T06:29:52.611 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:29:52.133+0000 7f883b4f4640 -1 osd.0 0 waiting for initial osdmap 2024-08-20T06:29:52.611 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:29:52.358+0000 7f8833313640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:29:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi107 2024-08-20T06:29:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi107 to 17087M 2024-08-20T06:29:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:29:52.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:52 smithi107 ceph-mon[18495]: from='osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884]' entity='osd.0' 2024-08-20T06:29:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:53 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:29:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:53 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:29:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:53 smithi183 ceph-mon[22204]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:53 smithi183 ceph-mon[22204]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:53.383 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:53 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:53 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:29:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:53 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:29:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:53 smithi193 ceph-mon[21944]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:53 smithi193 ceph-mon[21944]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:53 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:53 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:29:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:53 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:29:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:53 smithi107 ceph-mon[18495]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:53 smithi107 ceph-mon[18495]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:29:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:53 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] boot 2024-08-20T06:29:54.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: osdmap e8: 1 total, 1 up, 1 in 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/3143393201' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"}]': finished 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: osdmap e9: 2 total, 1 up, 2 in 2024-08-20T06:29:54.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] boot 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: osdmap e8: 1 total, 1 up, 1 in 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/3143393201' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"}]': finished 2024-08-20T06:29:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: osdmap e9: 2 total, 1 up, 2 in 2024-08-20T06:29:54.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: osd.0 [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] boot 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: osdmap e8: 1 total, 1 up, 1 in 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/3143393201' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"} : dispatch 2024-08-20T06:29:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7896e40d-68d0-4ca3-9a69-5364d3804903"}]': finished 2024-08-20T06:29:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: osdmap e9: 2 total, 1 up, 2 in 2024-08-20T06:29:54.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:29:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:55 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/2464198580' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:55.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:55 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/2464198580' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:55 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/2464198580' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:29:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:56 smithi193 ceph-mon[21944]: pgmap v56: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:29:56.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:56 smithi107 ceph-mon[18495]: pgmap v56: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:29:56.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:56 smithi183 ceph-mon[22204]: pgmap v56: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:29:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:29:58 smithi193 ceph-mon[21944]: pgmap v57: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:29:58.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:29:58 smithi107 ceph-mon[18495]: pgmap v57: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:29:58.632 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:29:58 smithi183 ceph-mon[22204]: pgmap v57: 0 pgs: ; 0 B data, 431 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:00.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:00 smithi183 ceph-mon[22204]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:00.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:00 smithi183 ceph-mon[22204]: overall HEALTH_OK 2024-08-20T06:30:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:00 smithi193 ceph-mon[21944]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:00 smithi193 ceph-mon[21944]: overall HEALTH_OK 2024-08-20T06:30:00.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:00 smithi107 ceph-mon[18495]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:00.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:00 smithi107 ceph-mon[18495]: overall HEALTH_OK 2024-08-20T06:30:02.431 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:02 smithi183 ceph-mon[22204]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:02.431 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T06:30:02.431 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:02.431 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:02 smithi183 ceph-mon[22204]: Deploying daemon osd.1 on smithi183 2024-08-20T06:30:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:02 smithi193 ceph-mon[21944]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T06:30:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:02 smithi193 ceph-mon[21944]: Deploying daemon osd.1 on smithi183 2024-08-20T06:30:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:02 smithi107 ceph-mon[18495]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T06:30:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:02 smithi107 ceph-mon[18495]: Deploying daemon osd.1 on smithi183 2024-08-20T06:30:04.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:04 smithi183 ceph-mon[22204]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:04.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:04 smithi193 ceph-mon[21944]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:04.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:04 smithi107 ceph-mon[18495]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:06.462 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:06 smithi183 ceph-mon[22204]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:06.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:06 smithi193 ceph-mon[21944]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:06.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:06 smithi107 ceph-mon[18495]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:07.447 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:07.447 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:07.447 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:08 smithi183 ceph-mon[22204]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:09.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:08 smithi193 ceph-mon[21944]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:09.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:08 smithi107 ceph-mon[18495]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:10.051 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 1 on host 'smithi183' 2024-08-20T06:30:10.052 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.050+0000 7f7755ffb640 1 -- 172.21.15.183:0/2131407010 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77580ff0a0 con 0x7f77480b3b40 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 msgr2=0x7f77480b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 0x7f77480b6000 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7744005fd0 tx=0x7f774400b390 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 msgr2=0x7f77581989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f774c00e9f0 tx=0x7f774c00eeb0 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 shutdown_connections 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7758103130 0x7f7758199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f77480b3b40 0x7f77480b6000 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7758102770 0x7f7758199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 --2- 172.21.15.183:0/2131407010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7758101610 0x7f77581989c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:10.054 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 >> 172.21.15.183:0/2131407010 conn(0x7f77580fcee0 msgr2=0x7f77580fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:30:10.057 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 shutdown_connections 2024-08-20T06:30:10.058 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:30:10.053+0000 7f775efd2640 1 -- 172.21.15.183:0/2131407010 wait complete. 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.404 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:10.718 DEBUG:teuthology.orchestra.run.smithi183:osd.1> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.1.service 2024-08-20T06:30:10.720 INFO:tasks.cephadm:Deploying osd.2 on smithi193 with /dev/vg_nvme/lv_4... 2024-08-20T06:30:10.720 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T06:30:10.974 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:30:11.423 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:11 smithi193 ceph-mon[21944]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:11.443 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:11 smithi183 ceph-mon[22204]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:11.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:11 smithi107 ceph-mon[18495]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:12.660 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:12 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:30:12.254+0000 7fd517307640 -1 osd.1 0 log_to_monitors true 2024-08-20T06:30:13.169 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:30:13.197 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-08-20T06:30:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:13 smithi183 ceph-mon[22204]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:13 smithi183 ceph-mon[22204]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:13 smithi183 ceph-mon[22204]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.452 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:30:13.487 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:13 smithi193 ceph-mon[21944]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:13.487 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:13 smithi193 ceph-mon[21944]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.487 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:13 smithi193 ceph-mon[21944]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:13 smithi107 ceph-mon[18495]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:13 smithi107 ceph-mon[18495]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:13 smithi107 ceph-mon[18495]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T06:30:13.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 -- 172.21.15.193:0/353440018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 msgr2=0x7f1634109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:13.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 --2- 172.21.15.193:0/353440018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f1634109ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f162c009cf0 tx=0x7f162c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 -- 172.21.15.193:0/353440018 shutdown_connections 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 --2- 172.21.15.193:0/353440018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f1634109ae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 --2- 172.21.15.193:0/353440018 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f16341027a0 0x7f1634102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 --2- 172.21.15.193:0/353440018 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1634101640 0x7f1634101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 -- 172.21.15.193:0/353440018 >> 172.21.15.193:0/353440018 conn(0x7f16340fcf30 msgr2=0x7f16340ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 -- 172.21.15.193:0/353440018 shutdown_connections 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.904+0000 7f1639a34640 1 -- 172.21.15.193:0/353440018 wait complete. 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 Processor -- start 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 -- start start 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1634101640 0x7f163406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16341027a0 0x7f163419d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:13.908 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f163410d250 con 0x7f1634101640 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f163410d0d0 con 0x7f16341027a0 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1639a34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f163410d3d0 con 0x7f1634103160 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f16337fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f16337fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:39944/0 (socket says 172.21.15.193:39944) 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f16337fe640 1 -- 172.21.15.193:0/2456455891 learned_addr learned my addr 172.21.15.193:0/2456455891 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f1632ffd640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1634101640 0x7f163406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.905+0000 7f16327fc640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16341027a0 0x7f163419d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 -- 172.21.15.193:0/2456455891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16341027a0 msgr2=0x7f163419d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:13.909 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16341027a0 0x7f163419d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 -- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1634101640 msgr2=0x7f163406bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1634101640 0x7f163406bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 -- 172.21.15.193:0/2456455891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16341a0800 con 0x7f1634103160 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f16337fe640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f162c031040 tx=0x7f162c003c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c003e60 con 0x7f1634103160 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16341a0a90 con 0x7f1634103160 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f162c040070 con 0x7f1634103160 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162c0058b0 con 0x7f1634103160 2024-08-20T06:30:13.910 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.906+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16341a1020 con 0x7f1634103160 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.908+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16341a1250 con 0x7f1634103160 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.909+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7f162c005050 con 0x7f1634103160 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.909+0000 7f1638a32640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 0x7f161c0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.910+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1903+0+0 (secure 0 0 0) 0x7f162c103480 con 0x7f1634103160 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.910+0000 7f1632ffd640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 0x7f161c0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.910+0000 7f1632ffd640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 0x7f161c0b6000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1624009990 tx=0x7f1624009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:30:13.914 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:13.914+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f162c02e050 con 0x7f1634103160 2024-08-20T06:30:14.103 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:14.103+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f16341a1420 con 0x7f161c0b3b40 2024-08-20T06:30:14.445 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T06:30:14.445 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: osdmap e10: 2 total, 1 up, 2 in 2024-08-20T06:30:14.445 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:14.445 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.445 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.446 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:14.447 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:14.447 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:14.486 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:30:14.196+0000 7fd51328a640 -1 osd.1 0 waiting for initial osdmap 2024-08-20T06:30:14.486 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:30:14.200+0000 7fd50f0b1640 -1 osd.1 11 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:30:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: osdmap e10: 2 total, 1 up, 2 in 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:14.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:14.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:14.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:14.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T06:30:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: osdmap e10: 2 total, 1 up, 2 in 2024-08-20T06:30:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:14.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi183 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi183 to 17088M 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:15.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:15.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi183 2024-08-20T06:30:15.518 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi183 to 17088M 2024-08-20T06:30:15.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:15.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:30:15.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T06:30:15.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:15.519 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: pgmap v66: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi183 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi183 to 17088M 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:16 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:30:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:16 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:30:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:16 smithi183 ceph-mon[22204]: osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] boot 2024-08-20T06:30:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:16 smithi183 ceph-mon[22204]: osdmap e12: 2 total, 2 up, 2 in 2024-08-20T06:30:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:16 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:16.524 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:16 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:30:16.524 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:16 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:30:16.524 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:16 smithi193 ceph-mon[21944]: osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] boot 2024-08-20T06:30:16.524 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:16 smithi193 ceph-mon[21944]: osdmap e12: 2 total, 2 up, 2 in 2024-08-20T06:30:16.524 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:16 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:16 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:30:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:16 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:30:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:16 smithi107 ceph-mon[18495]: osd.1 [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] boot 2024-08-20T06:30:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:16 smithi107 ceph-mon[18495]: osdmap e12: 2 total, 2 up, 2 in 2024-08-20T06:30:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:16 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: osdmap e13: 2 total, 2 up, 2 in 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/399936254' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"}]': finished 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: osdmap e14: 3 total, 2 up, 3 in 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:17 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/3971100110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: osdmap e13: 2 total, 2 up, 2 in 2024-08-20T06:30:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/399936254' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"}]': finished 2024-08-20T06:30:17.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: osdmap e14: 3 total, 2 up, 3 in 2024-08-20T06:30:17.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:17.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:17 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/3971100110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: osdmap e13: 2 total, 2 up, 2 in 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/399936254' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"} : dispatch 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "371c31f4-04e5-4bce-87c7-b8db82e15d1c"}]': finished 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: osdmap e14: 3 total, 2 up, 3 in 2024-08-20T06:30:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:17.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:17 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/3971100110' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:19 smithi183 ceph-mon[22204]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:19 smithi193 ceph-mon[21944]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:19.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:19 smithi107 ceph-mon[18495]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:21 smithi183 ceph-mon[22204]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:21 smithi193 ceph-mon[21944]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:21 smithi107 ceph-mon[18495]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:22.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:22 smithi193 ceph-mon[21944]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:22.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:22 smithi107 ceph-mon[18495]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:22.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:22 smithi183 ceph-mon[22204]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:24.330 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:24 smithi193 ceph-mon[21944]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:24.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:24 smithi107 ceph-mon[18495]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:24.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:24 smithi183 ceph-mon[22204]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:25.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T06:30:25.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:25.392 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:25 smithi193 ceph-mon[21944]: Deploying daemon osd.2 on smithi193 2024-08-20T06:30:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T06:30:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:25 smithi183 ceph-mon[22204]: Deploying daemon osd.2 on smithi193 2024-08-20T06:30:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T06:30:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:25 smithi107 ceph-mon[18495]: Deploying daemon osd.2 on smithi193 2024-08-20T06:30:26.515 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:26 smithi193 ceph-mon[21944]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:26.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:26 smithi107 ceph-mon[18495]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:26.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:26 smithi183 ceph-mon[22204]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:28.526 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:28 smithi193 ceph-mon[21944]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:28.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:28 smithi107 ceph-mon[18495]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:28 smithi183 ceph-mon[22204]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:30.418 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:30 smithi193 ceph-mon[21944]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:30.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:30.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:30.419 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:30 smithi107 ceph-mon[18495]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:30 smithi183 ceph-mon[22204]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:33.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:33.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:33.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:33.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:33.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:33.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:33.597 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.596+0000 7f1638a32640 1 -- 172.21.15.193:0/2456455891 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f16341a1420 con 0x7f161c0b3b40 2024-08-20T06:30:33.598 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 2 on host 'smithi193' 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 msgr2=0x7f161c0b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 0x7f161c0b6000 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1624009990 tx=0x7f1624009110 comp rx=0 tx=0).stop 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 msgr2=0x7f163419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f162c031040 tx=0x7f162c003c70 comp rx=0 tx=0).stop 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 shutdown_connections 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f161c0b3b40 0x7f161c0b6000 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1634103160 0x7f163419ff40 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f16341027a0 0x7f163419d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:33.600 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 --2- 172.21.15.193:0/2456455891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1634101640 0x7f163406bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:33.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.598+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 >> 172.21.15.193:0/2456455891 conn(0x7f16340fcf30 msgr2=0x7f1634103ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:30:33.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.599+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 shutdown_connections 2024-08-20T06:30:33.601 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:30:33.599+0000 7f1639a34640 1 -- 172.21.15.193:0/2456455891 wait complete. 2024-08-20T06:30:34.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:33 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:34.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:33 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.136 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:33 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.137 DEBUG:teuthology.orchestra.run.smithi193:osd.2> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.2.service 2024-08-20T06:30:34.140 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_3... 2024-08-20T06:30:34.140 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T06:30:34.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:33 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:34.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:33 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.168 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:33 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:33 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:34.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:33 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:33 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:34.395 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:30:35.134 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:34 smithi193 ceph-mon[21944]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:34 smithi183 ceph-mon[22204]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:34 smithi107 ceph-mon[18495]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:35.384 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:30:35.241+0000 7f205f5b5640 -1 osd.2 0 log_to_monitors true 2024-08-20T06:30:36.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:35 smithi107 ceph-mon[18495]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.119 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:35 smithi107 ceph-mon[18495]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.242 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:35 smithi183 ceph-mon[22204]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.242 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:35 smithi183 ceph-mon[22204]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:35 smithi193 ceph-mon[21944]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.310 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:35 smithi193 ceph-mon[21944]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T06:30:36.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:36.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T06:30:36.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T06:30:36.914 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:36.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:36.916 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:30:36.942 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-08-20T06:30:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:37.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:37.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: osdmap e15: 3 total, 2 up, 3 in 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:37.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 -- 172.21.15.107:0/4156175227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 msgr2=0x7fcb8c1052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 --2- 172.21.15.107:0/4156175227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1052f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009ca0 tx=0x7fcb7c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 -- 172.21.15.107:0/4156175227 shutdown_connections 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 --2- 172.21.15.107:0/4156175227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c10a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 --2- 172.21.15.107:0/4156175227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 --2- 172.21.15.107:0/4156175227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1052f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.665+0000 7fcb91787640 1 -- 172.21.15.107:0/4156175227 >> 172.21.15.107:0/4156175227 conn(0x7fcb8c0fcee0 msgr2=0x7fcb8c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- 172.21.15.107:0/4156175227 shutdown_connections 2024-08-20T06:30:37.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- 172.21.15.107:0/4156175227 wait complete. 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 Processor -- start 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- start start 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c10e2b0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c10e130 con 0x7fcb8c105880 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.666+0000 7fcb91787640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcb8c10e430 con 0x7fcb8c108430 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57392/0 (socket says 172.21.15.107:57392) 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 -- 172.21.15.107:0/492921365 learned_addr learned my addr 172.21.15.107:0/492921365 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:30:37.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8b7fe640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8a7fc640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 -- 172.21.15.107:0/492921365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 msgr2=0x7fcb8c199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 -- 172.21.15.107:0/492921365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 msgr2=0x7fcb8c199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 -- 172.21.15.107:0/492921365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb8c19a500 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8b7fe640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8a7fc640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb8affd640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c032ed0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb8c19a790 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcb7c0044d0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c030c80 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.667+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb8c1a0ed0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.668+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8c105ce0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.670+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) ==== 151004+0+0 (secure 0 0 0) 0x7fcb7c004040 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.671+0000 7fcb73fff640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 0x7fcb740b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.671+0000 7fcb8a7fc640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 0x7fcb740b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.671+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2571+0+0 (secure 0 0 0) 0x7fcb7c103be0 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.671+0000 7fcb8a7fc640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 0x7fcb740b6000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb800058c0 tx=0x7fcb80005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:30:37.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.674+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fcb7c0cc920 con 0x7fcb8c06b3e0 2024-08-20T06:30:37.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:37.859+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fcb8c0632c0 con 0x7fcb740b3b40 2024-08-20T06:30:38.054 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:30:37.794+0000 7f205bd4c640 -1 osd.2 0 waiting for initial osdmap 2024-08-20T06:30:38.054 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:30:37.954+0000 7f2053357640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:30:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi193 2024-08-20T06:30:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi193 to 17088M 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='osd.2 ' entity='osd.2' 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi193 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi193 to 17088M 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='osd.2 ' entity='osd.2' 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi193 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi193 to 17088M 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='osd.2 ' entity='osd.2' 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:30:38.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:30:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:30:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:38 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:39.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:38 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:39.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:30:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:30:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: from='client.14427 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:30:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:39.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:38 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:40.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:39 smithi183 ceph-mon[22204]: osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] boot 2024-08-20T06:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:39 smithi183 ceph-mon[22204]: osdmap e17: 3 total, 3 up, 3 in 2024-08-20T06:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:40.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:39 smithi107 ceph-mon[18495]: osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] boot 2024-08-20T06:30:40.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:39 smithi107 ceph-mon[18495]: osdmap e17: 3 total, 3 up, 3 in 2024-08-20T06:30:40.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:40.282 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:39 smithi193 ceph-mon[21944]: osd.2 [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] boot 2024-08-20T06:30:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:39 smithi193 ceph-mon[21944]: osdmap e17: 3 total, 3 up, 3 in 2024-08-20T06:30:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T06:30:40.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:40.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37277]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37277]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37285]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37285]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37285]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37285]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37270]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37270]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.564 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37273]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.565 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:30:40 smithi107 sudo[37273]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30510]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30510]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30510]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30510]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30513]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30513]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30513]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.736 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30513]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30516]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30516]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30516]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30516]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30519]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30519]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30519]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 sudo[30519]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30503]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30503]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30503]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30503]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30506]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30506]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30506]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30506]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30497]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30497]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30497]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30497]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30500]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T06:30:40.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30500]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T06:30:40.806 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30500]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T06:30:40.806 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:30:40 smithi193 sudo[30500]: pam_unix(sudo:session): session closed for user root 2024-08-20T06:30:41.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:41.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: osdmap e18: 3 total, 3 up, 3 in 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"} : dispatch 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"}]': finished 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: osdmap e19: 4 total, 3 up, 4 in 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:40 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2933449221' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: osdmap e18: 3 total, 3 up, 3 in 2024-08-20T06:30:41.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"} : dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"}]': finished 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: osdmap e19: 4 total, 3 up, 4 in 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:40 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2933449221' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:41.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-08-20T06:30:41.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: osdmap e18: 3 total, 3 up, 3 in 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"} : dispatch 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2892351012' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b8f09d-16d2-4c69-8d1e-d2835a148199"}]': finished 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: osdmap e19: 4 total, 3 up, 4 in 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T06:30:41.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:40 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2933449221' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:30:42.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:42.249+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fcb7c0c84b0 con 0x7fcb8c06b3e0 2024-08-20T06:30:42.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:42 smithi193 ceph-mon[21944]: osdmap e20: 4 total, 3 up, 4 in 2024-08-20T06:30:42.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:42 smithi107 ceph-mon[18495]: osdmap e20: 4 total, 3 up, 4 in 2024-08-20T06:30:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:42 smithi183 ceph-mon[22204]: osdmap e20: 4 total, 3 up, 4 in 2024-08-20T06:30:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:30:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:43 smithi193 ceph-mon[21944]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:43 smithi193 ceph-mon[21944]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-20T06:30:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:43 smithi107 ceph-mon[18495]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:43 smithi107 ceph-mon[18495]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-20T06:30:43.637 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:43 smithi183 ceph-mon[22204]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:43.637 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:43 smithi183 ceph-mon[22204]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-08-20T06:30:44.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:44 smithi193 ceph-mon[21944]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:44.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:44 smithi107 ceph-mon[18495]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:44.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:44 smithi183 ceph-mon[22204]: pgmap v91: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:46.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:46 smithi107 ceph-mon[18495]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:46.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:46 smithi193 ceph-mon[21944]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:46 smithi183 ceph-mon[22204]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:48.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:48 smithi107 ceph-mon[18495]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:48 smithi193 ceph-mon[21944]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:48 smithi183 ceph-mon[22204]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:49.421 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T06:30:49.421 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:49.421 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:49 smithi107 ceph-mon[18495]: Deploying daemon osd.3 on smithi107 2024-08-20T06:30:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T06:30:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:49 smithi193 ceph-mon[21944]: Deploying daemon osd.3 on smithi107 2024-08-20T06:30:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T06:30:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:49 smithi183 ceph-mon[22204]: Deploying daemon osd.3 on smithi107 2024-08-20T06:30:50.448 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:50 smithi107 ceph-mon[18495]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:50 smithi193 ceph-mon[21944]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:50 smithi183 ceph-mon[22204]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:52 smithi193 ceph-mon[21944]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:52 smithi107 ceph-mon[18495]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:52.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:52 smithi183 ceph-mon[22204]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:54 smithi193 ceph-mon[21944]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:54.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:54 smithi107 ceph-mon[18495]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:54.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:54.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:54.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:54.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:54 smithi183 ceph-mon[22204]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:56 smithi193 ceph-mon[21944]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:56.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:56 smithi107 ceph-mon[18495]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:56.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:56 smithi183 ceph-mon[22204]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:57.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:57.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:57.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:57.818 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:57.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.859+0000 7fcb73fff640 1 -- 172.21.15.107:0/492921365 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcb8c0632c0 con 0x7fcb740b3b40 2024-08-20T06:30:57.860 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-08-20T06:30:57.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 msgr2=0x7fcb740b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:57.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 0x7fcb740b6000 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb800058c0 tx=0x7fcb80005850 comp rx=0 tx=0).stop 2024-08-20T06:30:57.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 msgr2=0x7fcb8c1989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:30:57.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c006010 tx=0x7fcb7c003ba0 comp rx=0 tx=0).stop 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 shutdown_connections 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcb740b3b40 0x7fcb740b6000 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb8c108430 0x7fcb8c199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcb8c105880 0x7fcb8c199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 --2- 172.21.15.107:0/492921365 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcb8c06b3e0 0x7fcb8c1989d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.861+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 >> 172.21.15.107:0/492921365 conn(0x7fcb8c0fcee0 msgr2=0x7fcb8c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.862+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 shutdown_connections 2024-08-20T06:30:57.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:30:57.862+0000 7fcb91787640 1 -- 172.21.15.107:0/492921365 wait complete. 2024-08-20T06:30:57.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:30:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:30:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.473 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.3.service 2024-08-20T06:30:58.475 INFO:tasks.cephadm:Deploying osd.4 on smithi183 with /dev/vg_nvme/lv_3... 2024-08-20T06:30:58.476 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T06:30:58.729 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:30:58.780 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:58 smithi107 ceph-mon[18495]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:58.781 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:58.781 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.781 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:30:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:58 smithi193 ceph-mon[21944]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:30:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:58 smithi183 ceph-mon[22204]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:30:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:30:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:58.849 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:30:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:30:59.896 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:30:59 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:30:59.748+0000 7ff9bc841640 -1 osd.3 0 log_to_monitors true 2024-08-20T06:31:00.677 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:00 smithi183 ceph-mon[22204]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:00.677 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:00 smithi183 ceph-mon[22204]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T06:31:00.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:00 smithi193 ceph-mon[21944]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:00.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:00 smithi193 ceph-mon[21944]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T06:31:00.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:00 smithi107 ceph-mon[18495]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:00.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:00 smithi107 ceph-mon[18495]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T06:31:01.429 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:31:01.456 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_3 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: osdmap e21: 4 total, 3 up, 4 in 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi107 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.697 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi107 to 8543M 2024-08-20T06:31:01.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:01.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:01.698 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.709 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: osdmap e21: 4 total, 3 up, 4 in 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi107 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi107 to 8543M 2024-08-20T06:31:01.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:01.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:01.807 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.862 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:31:01.520+0000 7ff9b87c4640 -1 osd.3 0 waiting for initial osdmap 2024-08-20T06:31:01.862 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:31:01.526+0000 7ff9b45eb640 -1 osd.3 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:31:01.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T06:31:01.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: osdmap e21: 4 total, 3 up, 4 in 2024-08-20T06:31:01.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:31:01.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi107 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:01.863 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi107 to 8543M 2024-08-20T06:31:01.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:01.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:01.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:01.864 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:02.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- 172.21.15.183:0/2312351582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48102770 msgr2=0x7ffa48102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:02.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/2312351582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48102770 0x7ffa48102bf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009930 tx=0x7ffa3002a420 comp rx=0 tx=0).stop 2024-08-20T06:31:02.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- 172.21.15.183:0/2312351582 shutdown_connections 2024-08-20T06:31:02.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/2312351582 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48103130 0x7ffa48109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:02.146 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/2312351582 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48102770 0x7ffa48102bf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/2312351582 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa48101610 0x7ffa48101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- 172.21.15.183:0/2312351582 >> 172.21.15.183:0/2312351582 conn(0x7ffa480fcee0 msgr2=0x7ffa480ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- 172.21.15.183:0/2312351582 shutdown_connections 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- 172.21.15.183:0/2312351582 wait complete. 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 Processor -- start 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.131+0000 7ffa4d89f640 1 -- start start 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48102770 0x7ffa4819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa48103130 0x7ffa4819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4810d120 con 0x7ffa48102770 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4810cfa0 con 0x7ffa48101610 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4810d2a0 con 0x7ffa48103130 2024-08-20T06:31:02.147 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:48282/0 (socket says 172.21.15.183:48282) 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 -- 172.21.15.183:0/3364742425 learned_addr learned my addr 172.21.15.183:0/3364742425 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa3ffff640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48102770 0x7ffa4819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 -- 172.21.15.183:0/3364742425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa48103130 msgr2=0x7ffa4819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa48103130 0x7ffa4819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 -- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48102770 msgr2=0x7ffa4819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48102770 0x7ffa4819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 -- 172.21.15.183:0/3364742425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa481a04c0 con 0x7ffa48101610 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa46ffd640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c920 tx=0x7ffa3800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38007c70 con 0x7ffa48101610 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa38007e10 con 0x7ffa48101610 2024-08-20T06:31:02.148 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.132+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa48197440 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.133+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38018710 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.133+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa48197980 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.134+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa48108930 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.136+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7ffa3800f450 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.137+0000 7ffa4c89d640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 0x7ffa340b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.137+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3702+0+0 (secure 0 0 0) 0x7ffa380e2e30 con 0x7ffa48101610 2024-08-20T06:31:02.157 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.137+0000 7ffa3ffff640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 0x7ffa340b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:02.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.137+0000 7ffa3ffff640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 0x7ffa340b6090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009900 tx=0x7ffa30005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:02.158 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.139+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ffa38014030 con 0x7ffa48101610 2024-08-20T06:31:02.341 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:02.329+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ffa48198eb0 con 0x7ffa340b3bd0 2024-08-20T06:31:02.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:02.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:31:02.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T06:31:02.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] boot 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: osdmap e23: 4 total, 4 up, 4 in 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] boot 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: osdmap e23: 4 total, 4 up, 4 in 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:02.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-08-20T06:31:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:31:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: osdmap e22: 4 total, 3 up, 4 in 2024-08-20T06:31:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: osd.3 [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] boot 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: osdmap e23: 4 total, 4 up, 4 in 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:02.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:03.711 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:03 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:31:03.711 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:03 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:31:03.711 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:03 smithi183 ceph-mon[22204]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:03.711 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:03 smithi183 ceph-mon[22204]: osdmap e24: 4 total, 4 up, 4 in 2024-08-20T06:31:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:03 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:31:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:03 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:31:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:03 smithi193 ceph-mon[21944]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:03.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:03 smithi193 ceph-mon[21944]: osdmap e24: 4 total, 4 up, 4 in 2024-08-20T06:31:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:03 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:31:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:03 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:31:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:03 smithi107 ceph-mon[18495]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:03 smithi107 ceph-mon[18495]: osdmap e24: 4 total, 4 up, 4 in 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: osdmap e25: 4 total, 4 up, 4 in 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/1464719978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"}]': finished 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: osdmap e26: 5 total, 4 up, 5 in 2024-08-20T06:31:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:04 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:05.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:05.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: osdmap e25: 4 total, 4 up, 4 in 2024-08-20T06:31:05.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/1464719978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"}]': finished 2024-08-20T06:31:05.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: osdmap e26: 5 total, 4 up, 5 in 2024-08-20T06:31:05.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:04 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: osdmap e25: 4 total, 4 up, 4 in 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/1464719978' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"} : dispatch 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1047c4ef-e210-49ca-86d1-84d0b9af15e2"}]': finished 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: osdmap e26: 5 total, 4 up, 5 in 2024-08-20T06:31:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:04 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/614106471' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/614106471' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:06.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/614106471' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:06 smithi183 ceph-mon[22204]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-08-20T06:31:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:06 smithi193 ceph-mon[21944]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-08-20T06:31:07.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:06 smithi107 ceph-mon[18495]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-08-20T06:31:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:08 smithi183 ceph-mon[22204]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T06:31:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:08 smithi193 ceph-mon[21944]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T06:31:09.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:08 smithi107 ceph-mon[18495]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T06:31:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:10 smithi183 ceph-mon[22204]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:31:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:10 smithi193 ceph-mon[21944]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:31:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:10 smithi107 ceph-mon[18495]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:31:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:12 smithi183 ceph-mon[22204]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T06:31:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:12 smithi193 ceph-mon[21944]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T06:31:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:12 smithi107 ceph-mon[18495]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T06:31:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:14.246 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:13 smithi183 ceph-mon[22204]: Deploying daemon osd.4 on smithi183 2024-08-20T06:31:14.280 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:13 smithi193 ceph-mon[21944]: Deploying daemon osd.4 on smithi183 2024-08-20T06:31:14.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:13 smithi107 ceph-mon[18495]: Deploying daemon osd.4 on smithi183 2024-08-20T06:31:15.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:14 smithi183 ceph-mon[22204]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:31:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:14 smithi193 ceph-mon[21944]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:31:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:14 smithi107 ceph-mon[18495]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:31:17.202 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:16 smithi183 ceph-mon[22204]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T06:31:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:16 smithi193 ceph-mon[21944]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T06:31:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:16 smithi107 ceph-mon[18495]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T06:31:18.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:18.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:18.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:18.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:18.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:18.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:19.262 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:19 smithi107 ceph-mon[18495]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:19.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:19 smithi193 ceph-mon[21944]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:19.348 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:19 smithi183 ceph-mon[22204]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:21.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:21.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:21.047 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:21.048 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.085 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 4 on host 'smithi183' 2024-08-20T06:31:21.086 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.084+0000 7ffa4c89d640 1 -- 172.21.15.183:0/3364742425 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ffa48198eb0 con 0x7ffa340b3bd0 2024-08-20T06:31:21.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 msgr2=0x7ffa340b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:21.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 0x7ffa340b6090 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009900 tx=0x7ffa30005720 comp rx=0 tx=0).stop 2024-08-20T06:31:21.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 msgr2=0x7ffa4806b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:21.088 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800c920 tx=0x7ffa3800cde0 comp rx=0 tx=0).stop 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 shutdown_connections 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa340b3bd0 0x7ffa340b6090 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa48103130 0x7ffa4819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa48102770 0x7ffa4819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 --2- 172.21.15.183:0/3364742425 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa48101610 0x7ffa4806b0f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 >> 172.21.15.183:0/3364742425 conn(0x7ffa480fcee0 msgr2=0x7ffa48105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 shutdown_connections 2024-08-20T06:31:21.089 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:31:21.087+0000 7ffa4d89f640 1 -- 172.21.15.183:0/3364742425 wait complete. 2024-08-20T06:31:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:21.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:21.735 DEBUG:teuthology.orchestra.run.smithi183:osd.4> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.4.service 2024-08-20T06:31:21.738 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_3... 2024-08-20T06:31:21.739 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T06:31:21.991 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:31:22.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:22 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:22.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:22 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:22.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:22 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:22.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:22 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:22.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:22 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:22.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:22 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:22 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:22 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:22 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:23.168 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:23 smithi183 ceph-mon[22204]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:23.168 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:23 smithi193 ceph-mon[21944]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:23 smithi107 ceph-mon[18495]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:24.180 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:31:24.205 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-08-20T06:31:24.296 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:24 smithi183 ceph-mon[22204]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.296 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:24 smithi183 ceph-mon[22204]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.296 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:31:24 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:31:24.015+0000 7f279b9bf640 -1 osd.4 0 log_to_monitors true 2024-08-20T06:31:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:24 smithi107 ceph-mon[18495]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:24 smithi107 ceph-mon[18495]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.475 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:31:24.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:24 smithi193 ceph-mon[21944]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:24 smithi193 ceph-mon[21944]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T06:31:24.883 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 -- 172.21.15.193:0/3216271090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c103160 msgr2=0x7f5c6c109ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:24.884 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 --2- 172.21.15.193:0/3216271090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c103160 0x7f5c6c109ae0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009cf0 tx=0x7f5c6402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.884 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 -- 172.21.15.193:0/3216271090 shutdown_connections 2024-08-20T06:31:24.884 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 --2- 172.21.15.193:0/3216271090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c103160 0x7f5c6c109ae0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.884 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 --2- 172.21.15.193:0/3216271090 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 --2- 172.21.15.193:0/3216271090 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 0x7f5c6c101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.880+0000 7f5c73d17640 1 -- 172.21.15.193:0/3216271090 >> 172.21.15.193:0/3216271090 conn(0x7f5c6c0fcf30 msgr2=0x7f5c6c0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:31:24.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- 172.21.15.193:0/3216271090 shutdown_connections 2024-08-20T06:31:24.885 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- 172.21.15.193:0/3216271090 wait complete. 2024-08-20T06:31:24.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 Processor -- start 2024-08-20T06:31:24.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- start start 2024-08-20T06:31:24.886 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 0x7f5c6c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:24.887 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:24.887 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c103160 0x7f5c6c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:24.887 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c10d1c0 con 0x7f5c6c103160 2024-08-20T06:31:24.887 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c10d040 con 0x7f5c6c101640 2024-08-20T06:31:24.887 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c73d17640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c10d340 con 0x7f5c6c1027a0 2024-08-20T06:31:24.888 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.881+0000 7f5c712b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:24.888 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:52478/0 (socket says 172.21.15.193:52478) 2024-08-20T06:31:24.888 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 -- 172.21.15.193:0/2440262032 learned_addr learned my addr 172.21.15.193:0/2440262032 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-08-20T06:31:24.888 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c722b4640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c103160 0x7f5c6c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:24.888 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c71ab3640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 0x7f5c6c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 -- 172.21.15.193:0/2440262032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 msgr2=0x7f5c6c198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 0x7f5c6c198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 -- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c103160 msgr2=0x7f5c6c199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c103160 0x7f5c6c199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 -- 172.21.15.193:0/2440262032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c19a5e0 con 0x7f5c6c1027a0 2024-08-20T06:31:24.889 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c712b2640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d950 tx=0x7f5c5c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c017070 con 0x7f5c6c1027a0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c5c009c30 con 0x7f5c6c1027a0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c19a8d0 con 0x7f5c6c1027a0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c00f620 con 0x7f5c6c1027a0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.882+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c1a0fb0 con 0x7f5c6c1027a0 2024-08-20T06:31:24.890 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.883+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c1a1180 con 0x7f5c6c1027a0 2024-08-20T06:31:24.892 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.884+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f5c5c010460 con 0x7f5c6c1027a0 2024-08-20T06:31:24.892 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.885+0000 7f5c62ffd640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 0x7f5c580b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:24.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.885+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3805+0+0 (secure 0 0 0) 0x7f5c5c0e2670 con 0x7f5c6c1027a0 2024-08-20T06:31:24.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.886+0000 7f5c71ab3640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 0x7f5c580b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:24.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.886+0000 7f5c71ab3640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 0x7f5c580b6090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b60 tx=0x7f5c68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:24.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:24.889+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5c5c0aae60 con 0x7f5c6c1027a0 2024-08-20T06:31:25.080 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:25.080+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5c6c0ff150 con 0x7f5c580b3bd0 2024-08-20T06:31:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:25.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T06:31:25.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: osdmap e27: 5 total, 4 up, 5 in 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.307 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:25.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:25.308 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T06:31:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: osdmap e27: 5 total, 4 up, 5 in 2024-08-20T06:31:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:25.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:25.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:25.365 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: osdmap e27: 5 total, 4 up, 5 in 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:25.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:26.105 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:31:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:31:25.968+0000 7f2798156640 -1 osd.4 0 waiting for initial osdmap 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi183 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi183 to 8544M 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T06:31:26.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:26 smithi107 ceph-mon[18495]: from='osd.4 ' entity='osd.4' 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi183 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi183 to 8544M 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.362 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.363 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:26 smithi193 ceph-mon[21944]: from='osd.4 ' entity='osd.4' 2024-08-20T06:31:26.486 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:31:26.153+0000 7f278f761640 -1 osd.4 28 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:31:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi183 2024-08-20T06:31:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi183 to 8544M 2024-08-20T06:31:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: osdmap e28: 5 total, 4 up, 5 in 2024-08-20T06:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:26.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:26 smithi183 ceph-mon[22204]: from='osd.4 ' entity='osd.4' 2024-08-20T06:31:27.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:27 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:31:27.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:27 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:31:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:27 smithi193 ceph-mon[21944]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:27 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:27 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:27 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:31:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:27 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:31:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:27 smithi183 ceph-mon[22204]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:27 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:27 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:27 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:31:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:27 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:31:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:27 smithi107 ceph-mon[18495]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] boot 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: osdmap e29: 5 total, 5 up, 5 in 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/2049228394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"}]': finished 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: osdmap e30: 6 total, 5 up, 6 in 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:28 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/2281520123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:28.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] boot 2024-08-20T06:31:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: osdmap e29: 5 total, 5 up, 5 in 2024-08-20T06:31:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/2049228394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"}]': finished 2024-08-20T06:31:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: osdmap e30: 6 total, 5 up, 6 in 2024-08-20T06:31:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:28 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/2281520123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:28.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: osd.4 [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] boot 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: osdmap e29: 5 total, 5 up, 5 in 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/2049228394' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"} : dispatch 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e114967-5347-4fb6-a53b-db12e51cd2c1"}]': finished 2024-08-20T06:31:28.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: osdmap e30: 6 total, 5 up, 6 in 2024-08-20T06:31:28.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:28.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:28 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/2281520123' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:29 smithi183 ceph-mon[22204]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:29 smithi183 ceph-mon[22204]: osdmap e31: 6 total, 5 up, 6 in 2024-08-20T06:31:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:29 smithi193 ceph-mon[21944]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:29 smithi193 ceph-mon[21944]: osdmap e31: 6 total, 5 up, 6 in 2024-08-20T06:31:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:29 smithi107 ceph-mon[18495]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-08-20T06:31:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:29 smithi107 ceph-mon[18495]: osdmap e31: 6 total, 5 up, 6 in 2024-08-20T06:31:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:31 smithi107 ceph-mon[18495]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:31 smithi183 ceph-mon[22204]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:31 smithi193 ceph-mon[21944]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:33.373 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:33 smithi193 ceph-mon[21944]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:33 smithi183 ceph-mon[22204]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:33 smithi107 ceph-mon[18495]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:35.184 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:35 smithi193 ceph-mon[21944]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:35 smithi183 ceph-mon[22204]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:35.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:35 smithi107 ceph-mon[18495]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:36.214 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T06:31:36.214 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:36.214 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:36 smithi193 ceph-mon[21944]: Deploying daemon osd.5 on smithi193 2024-08-20T06:31:36.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T06:31:36.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:36.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:36 smithi183 ceph-mon[22204]: Deploying daemon osd.5 on smithi193 2024-08-20T06:31:36.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T06:31:36.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:36.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:36 smithi107 ceph-mon[18495]: Deploying daemon osd.5 on smithi193 2024-08-20T06:31:37.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:37 smithi193 ceph-mon[21944]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:37 smithi183 ceph-mon[22204]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:37.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:37 smithi107 ceph-mon[18495]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:39.455 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:39 smithi193 ceph-mon[21944]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:39 smithi183 ceph-mon[22204]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:39.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:39 smithi107 ceph-mon[18495]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:40.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:40.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:40.443 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:40 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:40.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:40 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:40 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:41 smithi183 ceph-mon[22204]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:41 smithi193 ceph-mon[21944]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:41.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:41 smithi107 ceph-mon[18495]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:43.424 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:43.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:43.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:43.425 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:43 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:43 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:43 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:43.699 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.697+0000 7f5c62ffd640 1 -- 172.21.15.193:0/2440262032 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c6c0ff150 con 0x7f5c580b3bd0 2024-08-20T06:31:43.699 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-08-20T06:31:43.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 msgr2=0x7f5c580b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:43.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 0x7f5c580b6090 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009b60 tx=0x7f5c68005f90 comp rx=0 tx=0).stop 2024-08-20T06:31:43.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 msgr2=0x7f5c6c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:43.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00d950 tx=0x7f5c5c00de10 comp rx=0 tx=0).stop 2024-08-20T06:31:43.701 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 shutdown_connections 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c6c103160 0x7f5c6c199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c580b3bd0 0x7f5c580b6090 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c6c1027a0 0x7f5c6c199100 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 --2- 172.21.15.193:0/2440262032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c6c101640 0x7f5c6c198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 >> 172.21.15.193:0/2440262032 conn(0x7f5c6c0fcf30 msgr2=0x7f5c6c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 shutdown_connections 2024-08-20T06:31:43.702 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:31:43.700+0000 7f5c73d17640 1 -- 172.21.15.193:0/2440262032 wait complete. 2024-08-20T06:31:44.346 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:44 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:44.346 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:44 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.346 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:44 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.373 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.5.service 2024-08-20T06:31:44.376 INFO:tasks.cephadm:Deploying osd.6 on smithi107 with /dev/vg_nvme/lv_2... 2024-08-20T06:31:44.376 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T06:31:44.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:44 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:44.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:44 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:44 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.478 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:31:44.479 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.479 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:44 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:44.629 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:31:45.319 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:45 smithi193 ceph-mon[21944]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:45 smithi107 ceph-mon[18495]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:45.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:45 smithi183 ceph-mon[22204]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:46.643 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:31:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:31:46.304+0000 7fa9bd077640 -1 osd.5 0 log_to_monitors true 2024-08-20T06:31:47.071 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:31:47.100 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-08-20T06:31:47.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:47.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.393 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:47.476 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:31:47.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- 172.21.15.107:0/2240090004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878101610 msgr2=0x7fb878101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 --2- 172.21.15.107:0/2240090004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878101610 0x7fb878101a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c02a420 comp rx=0 tx=0).stop 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- 172.21.15.107:0/2240090004 shutdown_connections 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 --2- 172.21.15.107:0/2240090004 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878103130 0x7fb878109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 --2- 172.21.15.107:0/2240090004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878102770 0x7fb878102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 --2- 172.21.15.107:0/2240090004 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878101610 0x7fb878101a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:47.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- 172.21.15.107:0/2240090004 >> 172.21.15.107:0/2240090004 conn(0x7fb8780fcee0 msgr2=0x7fb8780ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- 172.21.15.107:0/2240090004 shutdown_connections 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- 172.21.15.107:0/2240090004 wait complete. 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 Processor -- start 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.863+0000 7fb87eb08640 1 -- start start 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878101610 0x7fb87819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878103130 0x7fb87819be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb87810d190 con 0x7fb878102770 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb87810d010 con 0x7fb878101610 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87eb08640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb87810d310 con 0x7fb878103130 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37094/0 (socket says 172.21.15.107:37094) 2024-08-20T06:31:47.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 -- 172.21.15.107:0/1898833908 learned_addr learned my addr 172.21.15.107:0/1898833908 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87c8a4640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878101610 0x7fb87819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb87d0a5640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878103130 0x7fb87819be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 -- 172.21.15.107:0/1898833908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878103130 msgr2=0x7fb87819be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878103130 0x7fb87819be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 -- 172.21.15.107:0/1898833908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878101610 msgr2=0x7fb87819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878101610 0x7fb87819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 -- 172.21.15.107:0/1898833908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87819c710 con 0x7fb878102770 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.864+0000 7fb877fff640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c970 tx=0x7fb86400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.865+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864007bb0 con 0x7fb878102770 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.865+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb864007d50 con 0x7fb878102770 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.865+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87819ca00 con 0x7fb878102770 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.865+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864004e40 con 0x7fb878102770 2024-08-20T06:31:47.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.865+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a30e0 con 0x7fb878102770 2024-08-20T06:31:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.866+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb878101a10 con 0x7fb878102770 2024-08-20T06:31:47.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.868+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fb864005000 con 0x7fb878102770 2024-08-20T06:31:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.869+0000 7fb875ffb640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 0x7fb8680b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:31:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.869+0000 7fb87c8a4640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 0x7fb8680b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:31:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.869+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4237+0+0 (secure 0 0 0) 0x7fb8640e2c80 con 0x7fb878102770 2024-08-20T06:31:47.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.869+0000 7fb87c8a4640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 0x7fb8680b6090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005ab0 tx=0x7fb86c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:31:47.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:47.871+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb864014030 con 0x7fb878102770 2024-08-20T06:31:48.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:31:48.064+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb8780ff050 con 0x7fb8680b3bd0 2024-08-20T06:31:48.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi193 2024-08-20T06:31:48.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi193 to 8544M 2024-08-20T06:31:48.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T06:31:48.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: osdmap e32: 6 total, 5 up, 6 in 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T06:31:48.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi193 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi193 to 8544M 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: osdmap e32: 6 total, 5 up, 6 in 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:48.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:31:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T06:31:48.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:48.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:31:48.163+0000 7fa9b8ffa640 -1 osd.5 0 waiting for initial osdmap 2024-08-20T06:31:48.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:31:48.171+0000 7fa9b4e21640 -1 osd.5 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:31:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi193 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi193 to 8544M 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: osdmap e32: 6 total, 5 up, 6 in 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994]' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: osdmap e33: 6 total, 5 up, 6 in 2024-08-20T06:31:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.311 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:49 smithi107 ceph-mon[18495]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:49.311 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:49 smithi107 ceph-mon[18495]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:49.311 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.311 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:49 smithi183 ceph-mon[22204]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:49 smithi183 ceph-mon[22204]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:49 smithi193 ceph-mon[21944]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:49 smithi193 ceph-mon[21944]: from='client.14505 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:31:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:50 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:31:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:50 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:31:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:50 smithi107 ceph-mon[18495]: osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] boot 2024-08-20T06:31:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:50 smithi107 ceph-mon[18495]: osdmap e34: 6 total, 6 up, 6 in 2024-08-20T06:31:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:50 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:31:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:50 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:31:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:50 smithi183 ceph-mon[22204]: osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] boot 2024-08-20T06:31:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:50 smithi183 ceph-mon[22204]: osdmap e34: 6 total, 6 up, 6 in 2024-08-20T06:31:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:50 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:31:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:50 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:31:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:50 smithi193 ceph-mon[21944]: osd.5 [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] boot 2024-08-20T06:31:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:50 smithi193 ceph-mon[21944]: osdmap e34: 6 total, 6 up, 6 in 2024-08-20T06:31:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: osdmap e35: 6 total, 6 up, 6 in 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"} : dispatch 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"}]': finished 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: osdmap e36: 7 total, 6 up, 7 in 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:31:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3228241170' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: osdmap e35: 6 total, 6 up, 6 in 2024-08-20T06:31:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"} : dispatch 2024-08-20T06:31:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"}]': finished 2024-08-20T06:31:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: osdmap e36: 7 total, 6 up, 7 in 2024-08-20T06:31:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:31:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3228241170' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: osdmap e35: 6 total, 6 up, 6 in 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"} : dispatch 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3358888021' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56c0c67a-ffc9-49e3-976f-4a1c19cc6786"}]': finished 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: osdmap e36: 7 total, 6 up, 7 in 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:31:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3228241170' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:31:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:53 smithi183 ceph-mon[22204]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:53 smithi193 ceph-mon[21944]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:53 smithi107 ceph-mon[18495]: pgmap v141: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:55 smithi183 ceph-mon[22204]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:55 smithi193 ceph-mon[21944]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:55.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:55 smithi107 ceph-mon[18495]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:31:57.484 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:57 smithi107 ceph-mon[18495]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-20T06:31:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:57 smithi183 ceph-mon[22204]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-20T06:31:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:57 smithi193 ceph-mon[21944]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-08-20T06:31:59.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:59 smithi107 ceph-mon[18495]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:59.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T06:31:59.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:31:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:59 smithi183 ceph-mon[22204]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T06:31:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:31:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:31:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:59 smithi193 ceph-mon[21944]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:31:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T06:31:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:31:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:00.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:00 smithi107 ceph-mon[18495]: Deploying daemon osd.6 on smithi107 2024-08-20T06:32:00.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:00 smithi183 ceph-mon[22204]: Deploying daemon osd.6 on smithi107 2024-08-20T06:32:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:00 smithi193 ceph-mon[21944]: Deploying daemon osd.6 on smithi107 2024-08-20T06:32:01.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:01 smithi107 ceph-mon[18495]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-20T06:32:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:01 smithi183 ceph-mon[22204]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-20T06:32:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:01 smithi193 ceph-mon[21944]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-08-20T06:32:03.437 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:03 smithi107 ceph-mon[18495]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:32:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:03 smithi183 ceph-mon[22204]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:32:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:03 smithi193 ceph-mon[21944]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:32:05.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:05 smithi107 ceph-mon[18495]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:05.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:05.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:05.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:05 smithi183 ceph-mon[22204]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:05 smithi193 ceph-mon[21944]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:07.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:07 smithi107 ceph-mon[18495]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:07 smithi183 ceph-mon[22204]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:07.554 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 6 on host 'smithi107' 2024-08-20T06:32:07.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.552+0000 7fb875ffb640 1 -- 172.21.15.107:0/1898833908 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb8780ff050 con 0x7fb8680b3bd0 2024-08-20T06:32:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:07 smithi193 ceph-mon[21944]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:32:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 msgr2=0x7fb8680b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 0x7fb8680b6090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005ab0 tx=0x7fb86c005a20 comp rx=0 tx=0).stop 2024-08-20T06:32:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 msgr2=0x7fb87819b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:07.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb86400c970 tx=0x7fb86400ce30 comp rx=0 tx=0).stop 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 shutdown_connections 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb8680b3bd0 0x7fb8680b6090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb878103130 0x7fb87819be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb878102770 0x7fb87819b230 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 --2- 172.21.15.107:0/1898833908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb878101610 0x7fb87819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 >> 172.21.15.107:0/1898833908 conn(0x7fb8780fcee0 msgr2=0x7fb8780fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 shutdown_connections 2024-08-20T06:32:07.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:07.555+0000 7fb87eb08640 1 -- 172.21.15.107:0/1898833908 wait complete. 2024-08-20T06:32:08.205 DEBUG:teuthology.orchestra.run.smithi107:osd.6> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.6.service 2024-08-20T06:32:08.207 INFO:tasks.cephadm:Deploying osd.7 on smithi183 with /dev/vg_nvme/lv_2... 2024-08-20T06:32:08.208 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T06:32:08.463 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:32:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:08 smithi193 ceph-mon[21944]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:08 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:08 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:08 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:08 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:08 smithi183 ceph-mon[22204]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:08.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:08 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:08 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.844 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:08 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.845 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:08 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.857 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:08 smithi107 ceph-mon[18495]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:08.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:08 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:08 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:08 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:08.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:08 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:10.361 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:32:10 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:32:10.088+0000 7f5dcf3e5640 -1 osd.6 0 log_to_monitors true 2024-08-20T06:32:10.768 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:10 smithi183 ceph-mon[22204]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:10.768 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:10 smithi183 ceph-mon[22204]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T06:32:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:10 smithi193 ceph-mon[21944]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:10 smithi193 ceph-mon[21944]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T06:32:10.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:10 smithi107 ceph-mon[18495]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:10.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:10 smithi107 ceph-mon[18495]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T06:32:10.950 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:32:10.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_2 2024-08-20T06:32:11.222 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:32:11.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T06:32:11.596 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: osdmap e37: 7 total, 6 up, 7 in 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.597 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:11.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:11.598 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 -- 172.21.15.183:0/114822642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678105880 msgr2=0x7f7678107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 --2- 172.21.15.183:0/114822642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678105880 0x7f7678107cf0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402a410 comp rx=0 tx=0).stop 2024-08-20T06:32:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 -- 172.21.15.183:0/114822642 shutdown_connections 2024-08-20T06:32:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 --2- 172.21.15.183:0/114822642 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678108430 0x7f767810a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:11.681 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 --2- 172.21.15.183:0/114822642 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678105880 0x7f7678107cf0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 --2- 172.21.15.183:0/114822642 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 0x7f76781052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.677+0000 7f768035a640 1 -- 172.21.15.183:0/114822642 >> 172.21.15.183:0/114822642 conn(0x7f76780fcee0 msgr2=0x7f76780ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- 172.21.15.183:0/114822642 shutdown_connections 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- 172.21.15.183:0/114822642 wait complete. 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 Processor -- start 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- start start 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 0x7f7678102a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678105880 0x7f7678101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:11.682 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e210 con 0x7f7678105880 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e090 con 0x7f7678108430 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.678+0000 7f768035a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f767810e390 con 0x7f767806b3e0 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:46426/0 (socket says 172.21.15.183:46426) 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 -- 172.21.15.183:0/1295463187 learned_addr learned my addr 172.21.15.183:0/1295463187 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e0f6640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 0x7f7678102a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767d8f5640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678105880 0x7f7678101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:11.683 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 -- 172.21.15.183:0/1295463187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 msgr2=0x7f7678102a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 0x7f7678102a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 -- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678105880 msgr2=0x7f7678101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678105880 0x7f7678101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 -- 172.21.15.183:0/1295463187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7678102090 con 0x7f7678108430 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f767e8f7640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7670009fd0 tx=0x7f767000def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670017070 con 0x7f7678108430 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76781a2ac0 con 0x7f7678108430 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7670003e00 con 0x7f7678108430 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670010460 con 0x7f7678108430 2024-08-20T06:32:11.684 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.679+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a2fd0 con 0x7f7678108430 2024-08-20T06:32:11.685 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.680+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76780fe9d0 con 0x7f7678108430 2024-08-20T06:32:11.685 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.682+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f7670003970 con 0x7f7678108430 2024-08-20T06:32:11.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.683+0000 7f766f7fe640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 0x7f76640b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:11.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.683+0000 7f767e0f6640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 0x7f76640b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:11.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.683+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4685+0+0 (secure 0 0 0) 0x7f76700e2a60 con 0x7f7678108430 2024-08-20T06:32:11.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.683+0000 7f767e0f6640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 0x7f76640b6090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f766800abe0 tx=0x7f766800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:11.686 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.686+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f76700aaee0 con 0x7f7678108430 2024-08-20T06:32:11.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:32:11.568+0000 7f5dcb368640 -1 osd.6 0 waiting for initial osdmap 2024-08-20T06:32:11.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:32:11.575+0000 7f5dc718f640 -1 osd.6 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:32:11.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T06:32:11.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: osdmap e37: 7 total, 6 up, 7 in 2024-08-20T06:32:11.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:11.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:11.872 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:11.872+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f76780632c0 con 0x7f76640b3bd0 2024-08-20T06:32:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T06:32:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: osdmap e37: 7 total, 6 up, 7 in 2024-08-20T06:32:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:12.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi107 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi107 to 5695M 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T06:32:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:12.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi107 2024-08-20T06:32:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi107 to 5695M 2024-08-20T06:32:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:32:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:12.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:13.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-08-20T06:32:13.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi107 2024-08-20T06:32:13.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi107 to 5695M 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: osdmap e38: 7 total, 6 up, 7 in 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:13.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:13 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:13 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:13 smithi183 ceph-mon[22204]: osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] boot 2024-08-20T06:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:13 smithi183 ceph-mon[22204]: osdmap e39: 7 total, 7 up, 7 in 2024-08-20T06:32:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:13 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:32:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:13 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:32:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:13 smithi107 ceph-mon[18495]: osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] boot 2024-08-20T06:32:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:13 smithi107 ceph-mon[18495]: osdmap e39: 7 total, 7 up, 7 in 2024-08-20T06:32:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:13.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:13 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:32:13.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:13 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:32:13.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:13 smithi193 ceph-mon[21944]: osd.6 [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] boot 2024-08-20T06:32:13.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:13 smithi193 ceph-mon[21944]: osdmap e39: 7 total, 7 up, 7 in 2024-08-20T06:32:13.956 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T06:32:14.711 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: osdmap e40: 7 total, 7 up, 7 in 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/228212507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"}]': finished 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: osdmap e41: 8 total, 7 up, 8 in 2024-08-20T06:32:14.712 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:14.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: osdmap e40: 7 total, 7 up, 7 in 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/228212507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"}]': finished 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: osdmap e41: 8 total, 7 up, 8 in 2024-08-20T06:32:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:15.057 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: osdmap e40: 7 total, 7 up, 7 in 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/228212507' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"} : dispatch 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2ed828c5-26a6-488d-a45a-18937c0520cb"}]': finished 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: osdmap e41: 8 total, 7 up, 8 in 2024-08-20T06:32:15.058 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:15.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:15 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/4076428174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:15.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:15 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/4076428174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:16.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:15 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/4076428174' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:16 smithi107 ceph-mon[18495]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:16.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:16 smithi183 ceph-mon[22204]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:17.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:16 smithi193 ceph-mon[21944]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:18.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:18 smithi107 ceph-mon[18495]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:18.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:18 smithi183 ceph-mon[22204]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:19.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:18 smithi193 ceph-mon[21944]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:20.793 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:20 smithi183 ceph-mon[22204]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:20 smithi107 ceph-mon[18495]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:21.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:20 smithi193 ceph-mon[21944]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:22.856 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:22 smithi107 ceph-mon[18495]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:22.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:22 smithi183 ceph-mon[22204]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:23.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:22 smithi193 ceph-mon[21944]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:23.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T06:32:23.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:23.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:23 smithi107 ceph-mon[18495]: Deploying daemon osd.7 on smithi183 2024-08-20T06:32:23.954 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T06:32:23.955 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:23.955 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:23 smithi183 ceph-mon[22204]: Deploying daemon osd.7 on smithi183 2024-08-20T06:32:24.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T06:32:24.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:24.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:23 smithi193 ceph-mon[21944]: Deploying daemon osd.7 on smithi183 2024-08-20T06:32:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:24 smithi107 ceph-mon[18495]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:24.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:24 smithi183 ceph-mon[22204]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:25.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:24 smithi193 ceph-mon[21944]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:26.855 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:26 smithi183 ceph-mon[22204]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:27.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:26 smithi193 ceph-mon[21944]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:27.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:26 smithi107 ceph-mon[18495]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:28 smithi183 ceph-mon[22204]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:28 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:28 smithi193 ceph-mon[21944]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:28 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:28 smithi107 ceph-mon[18495]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:28 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.004 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.004+0000 7f766f7fe640 1 -- 172.21.15.183:0/1295463187 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f76780632c0 con 0x7f76640b3bd0 2024-08-20T06:32:31.005 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 7 on host 'smithi183' 2024-08-20T06:32:31.006 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 msgr2=0x7f76640b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 0x7f76640b6090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f766800abe0 tx=0x7f766800a2d0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 msgr2=0x7f76781017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7670009fd0 tx=0x7f767000def0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 shutdown_connections 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7678108430 0x7f76781017d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7678105880 0x7f7678101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f76640b3bd0 0x7f76640b6090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 --2- 172.21.15.183:0/1295463187 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f767806b3e0 0x7f7678102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 >> 172.21.15.183:0/1295463187 conn(0x7f76780fcee0 msgr2=0x7f7678103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 shutdown_connections 2024-08-20T06:32:31.007 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:32:31.006+0000 7f768035a640 1 -- 172.21.15.183:0/1295463187 wait complete. 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:31 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:31 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.545 DEBUG:teuthology.orchestra.run.smithi183:osd.7> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.7.service 2024-08-20T06:32:31.547 INFO:tasks.cephadm:Deploying osd.8 on smithi193 with /dev/vg_nvme/lv_2... 2024-08-20T06:32:31.548 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:31 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:31.805 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:32:33.187 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:33 smithi193 ceph-mon[21944]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:33 smithi183 ceph-mon[22204]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:33 smithi107 ceph-mon[18495]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:33.839 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:32:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:32:33.750+0000 7f603fe4a640 -1 osd.7 0 log_to_monitors true 2024-08-20T06:32:34.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:34 smithi183 ceph-mon[22204]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.156 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:34 smithi183 ceph-mon[22204]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.294 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:34 smithi193 ceph-mon[21944]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:34 smithi193 ceph-mon[21944]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.297 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:32:34.328 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-08-20T06:32:34.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:34 smithi107 ceph-mon[18495]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:34 smithi107 ceph-mon[18495]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T06:32:34.621 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:32:35.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 -- 172.21.15.193:0/1038013968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc406b340 msgr2=0x7f3bc4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 --2- 172.21.15.193:0/1038013968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc406b340 0x7f3bc4105390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3bb400dc90 tx=0x7f3bb402e370 comp rx=0 tx=0).stop 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 -- 172.21.15.193:0/1038013968 shutdown_connections 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 --2- 172.21.15.193:0/1038013968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 0x7f3bc410a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 --2- 172.21.15.193:0/1038013968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc41058d0 0x7f3bc4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 --2- 172.21.15.193:0/1038013968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc406b340 0x7f3bc4105390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 -- 172.21.15.193:0/1038013968 >> 172.21.15.193:0/1038013968 conn(0x7f3bc40fcee0 msgr2=0x7f3bc40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 -- 172.21.15.193:0/1038013968 shutdown_connections 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 -- 172.21.15.193:0/1038013968 wait complete. 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.042+0000 7f3bca925640 1 Processor -- start 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 -- start start 2024-08-20T06:32:35.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc406b340 0x7f3bc4196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 0x7f3bc4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410c0f0 con 0x7f3bc4108420 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410bf70 con 0x7f3bc406b340 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bca925640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc410c270 con 0x7f3bc41058d0 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:54890/0 (socket says 172.21.15.193:54890) 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 -- 172.21.15.193:0/333097060 learned_addr learned my addr 172.21.15.193:0/333097060 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc3fff640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc406b340 0x7f3bc4196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc8ec2640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 0x7f3bc4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 -- 172.21.15.193:0/333097060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc406b340 msgr2=0x7f3bc4196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:35.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc406b340 0x7f3bc4196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 -- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 msgr2=0x7f3bc4197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 0x7f3bc4197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.043+0000 7f3bc37fe640 1 -- 172.21.15.193:0/333097060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4198480 con 0x7f3bc41058d0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bc37fe640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b4a0 tx=0x7f3bb800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8014070 con 0x7f3bc41058d0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3bb8010aa0 con 0x7f3bc41058d0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc4198770 con 0x7f3bc41058d0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8017b70 con 0x7f3bc41058d0 2024-08-20T06:32:35.047 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.044+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc419ee50 con 0x7f3bc41058d0 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.045+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc40fe9d0 con 0x7f3bc41058d0 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.046+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f3bb8017070 con 0x7f3bc41058d0 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.047+0000 7f3bc17fa640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 0x7f3bac0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.047+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5101+0+0 (secure 0 0 0) 0x7f3bb80e2630 con 0x7f3bc41058d0 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.047+0000 7f3bc3fff640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 0x7f3bac0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:35.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.048+0000 7f3bc3fff640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 0x7f3bac0b6090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40057c0 tx=0x7f3bb4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:35.050 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.050+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3bb80aa910 con 0x7f3bc41058d0 2024-08-20T06:32:35.244 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:35.244+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3bc40632c0 con 0x7f3bac0b3bd0 2024-08-20T06:32:35.244 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: osdmap e42: 8 total, 7 up, 8 in 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi183 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.245 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi183 to 5696M 2024-08-20T06:32:35.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:35.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:35.246 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: osdmap e42: 8 total, 7 up, 8 in 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi183 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi183 to 5696M 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:35.364 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: osdmap e42: 8 total, 7 up, 8 in 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi183 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi183 to 5696M 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:35.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:36.236 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:32:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:32:35.973+0000 7f603bdcd640 -1 osd.7 0 waiting for initial osdmap 2024-08-20T06:32:36.236 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:32:36.179+0000 7f6033bec640 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:32:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:32:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T06:32:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='osd.7 ' entity='osd.7' 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T06:32:36.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:32:36.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T06:32:36.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='osd.7 ' entity='osd.7' 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T06:32:36.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: osdmap e43: 8 total, 7 up, 8 in 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='osd.7 ' entity='osd.7' 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: osdmap e44: 8 total, 7 up, 8 in 2024-08-20T06:32:36.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.222 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:32:37.222 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:32:37.222 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:37.222 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] boot 2024-08-20T06:32:37.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: osdmap e45: 8 total, 8 up, 8 in 2024-08-20T06:32:37.223 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] boot 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: osdmap e45: 8 total, 8 up, 8 in 2024-08-20T06:32:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: pgmap v170: 1 pgs: 1 unknown; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: osd.7 [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] boot 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: osdmap e45: 8 total, 8 up, 8 in 2024-08-20T06:32:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T06:32:38.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:38 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/845011303' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:38 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:38 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"}]': finished 2024-08-20T06:32:38.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:38 smithi107 ceph-mon[18495]: osdmap e46: 9 total, 8 up, 9 in 2024-08-20T06:32:38.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:38 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:38 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/845011303' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:38 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:38 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"}]': finished 2024-08-20T06:32:38.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:38 smithi183 ceph-mon[22204]: osdmap e46: 9 total, 8 up, 9 in 2024-08-20T06:32:38.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:38 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:38 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/845011303' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:38 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"} : dispatch 2024-08-20T06:32:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:38 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1f41a3e6-90c8-46c8-bcd6-883e5b7a253a"}]': finished 2024-08-20T06:32:38.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:38 smithi193 ceph-mon[21944]: osdmap e46: 9 total, 8 up, 9 in 2024-08-20T06:32:38.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:38 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:39 smithi107 ceph-mon[18495]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:39 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/2785473839' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:39 smithi107 ceph-mon[18495]: osdmap e47: 9 total, 8 up, 9 in 2024-08-20T06:32:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:39 smithi183 ceph-mon[22204]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:39 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/2785473839' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:39 smithi183 ceph-mon[22204]: osdmap e47: 9 total, 8 up, 9 in 2024-08-20T06:32:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:39 smithi193 ceph-mon[21944]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:39 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/2785473839' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:32:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:39 smithi193 ceph-mon[21944]: osdmap e47: 9 total, 8 up, 9 in 2024-08-20T06:32:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:41.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:41 smithi107 ceph-mon[18495]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:41 smithi183 ceph-mon[22204]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:41 smithi193 ceph-mon[21944]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:43.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:43 smithi107 ceph-mon[18495]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:43.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:43 smithi183 ceph-mon[22204]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:43 smithi193 ceph-mon[21944]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 236 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:45 smithi183 ceph-mon[22204]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-08-20T06:32:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:45 smithi193 ceph-mon[21944]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-08-20T06:32:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:45 smithi107 ceph-mon[18495]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 187 KiB/s, 0 objects/s recovering 2024-08-20T06:32:46.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T06:32:46.376 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:46.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T06:32:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T06:32:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:47.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:47 smithi193 ceph-mon[21944]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T06:32:47.377 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:47 smithi193 ceph-mon[21944]: Deploying daemon osd.8 on smithi193 2024-08-20T06:32:47.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:47 smithi183 ceph-mon[22204]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T06:32:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:47 smithi183 ceph-mon[22204]: Deploying daemon osd.8 on smithi193 2024-08-20T06:32:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:47 smithi107 ceph-mon[18495]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-08-20T06:32:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:47 smithi107 ceph-mon[18495]: Deploying daemon osd.8 on smithi193 2024-08-20T06:32:49.238 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:49 smithi193 ceph-mon[21944]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-20T06:32:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:49 smithi183 ceph-mon[22204]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-20T06:32:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:49 smithi107 ceph-mon[18495]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-08-20T06:32:51.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:51 smithi193 ceph-mon[21944]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-08-20T06:32:51.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:51.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:51.379 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:51.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:51 smithi107 ceph-mon[18495]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-08-20T06:32:51.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:51.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:51.413 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:51 smithi183 ceph-mon[22204]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-08-20T06:32:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:53 smithi183 ceph-mon[22204]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:53 smithi193 ceph-mon[21944]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:53 smithi107 ceph-mon[18495]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:54.253 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.251+0000 7f3bc17fa640 1 -- 172.21.15.193:0/333097060 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bc40632c0 con 0x7f3bac0b3bd0 2024-08-20T06:32:54.253 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 8 on host 'smithi193' 2024-08-20T06:32:54.255 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.253+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 msgr2=0x7f3bac0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.253+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 0x7f3bac0b6090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3bb40057c0 tx=0x7f3bb4000bc0 comp rx=0 tx=0).stop 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.253+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 msgr2=0x7f3bc4196fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.253+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800b4a0 tx=0x7f3bb800b960 comp rx=0 tx=0).stop 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 shutdown_connections 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3bc4108420 0x7f3bc4197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3bac0b3bd0 0x7f3bac0b6090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:54.257 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bc41058d0 0x7f3bc4196fa0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:54.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 --2- 172.21.15.193:0/333097060 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3bc406b340 0x7f3bc4196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:54.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 >> 172.21.15.193:0/333097060 conn(0x7f3bc40fcee0 msgr2=0x7f3bc4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:54.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 shutdown_connections 2024-08-20T06:32:54.258 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:32:54.254+0000 7f3bca925640 1 -- 172.21.15.193:0/333097060 wait complete. 2024-08-20T06:32:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:54.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:54.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:54.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:54.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:54.923 DEBUG:teuthology.orchestra.run.smithi193:osd.8> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.8.service 2024-08-20T06:32:54.926 INFO:tasks.cephadm:Deploying osd.9 on smithi107 with /dev/vg_nvme/lv_1... 2024-08-20T06:32:54.926 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T06:32:55.181 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:32:55.532 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:55 smithi107 ceph-mon[18495]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:55.532 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:55.532 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:55.533 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:55 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:55 smithi193 ceph-mon[21944]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:55 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:55 smithi183 ceph-mon[22204]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T06:32:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:32:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:55 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:56.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:32:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:32:56.419+0000 7f3225492640 -1 osd.8 0 log_to_monitors true 2024-08-20T06:32:57.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:57 smithi107 ceph-mon[18495]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:57.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:57 smithi107 ceph-mon[18495]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:57 smithi107 ceph-mon[18495]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:57 smithi193 ceph-mon[21944]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:57 smithi193 ceph-mon[21944]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:57 smithi193 ceph-mon[21944]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:57 smithi183 ceph-mon[22204]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:57 smithi183 ceph-mon[22204]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:57 smithi183 ceph-mon[22204]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T06:32:57.905 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:32:57.929 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-08-20T06:32:58.175 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:32:58.538 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: osdmap e48: 9 total, 8 up, 9 in 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:58.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:58.540 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:58 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T06:32:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: osdmap e48: 9 total, 8 up, 9 in 2024-08-20T06:32:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:58 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 -- 172.21.15.107:0/736065135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 msgr2=0x7f9f7c101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 --2- 172.21.15.107:0/736065135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c101a40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009930 tx=0x7f9f6c02a420 comp rx=0 tx=0).stop 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 -- 172.21.15.107:0/736065135 shutdown_connections 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 --2- 172.21.15.107:0/736065135 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c103160 0x7f9f7c109ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 --2- 172.21.15.107:0/736065135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c1027a0 0x7f9f7c102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 --2- 172.21.15.107:0/736065135 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c101a40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:58.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 -- 172.21.15.107:0/736065135 >> 172.21.15.107:0/736065135 conn(0x7f9f7c0fcf30 msgr2=0x7f9f7c0ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 -- 172.21.15.107:0/736065135 shutdown_connections 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.621+0000 7f9f8242e640 1 -- 172.21.15.107:0/736065135 wait complete. 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 Processor -- start 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 -- start start 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c1027a0 0x7f9f7c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c103160 0x7f9f7c199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10e2e0 con 0x7f9f7c101640 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10e160 con 0x7f9f7c1027a0 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f8242e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7c10e460 con 0x7f9f7c103160 2024-08-20T06:32:58.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44232/0 (socket says 172.21.15.107:44232) 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f7bfff640 1 -- 172.21.15.107:0/662601612 learned_addr learned my addr 172.21.15.107:0/662601612 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f809cb640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c103160 0x7f9f7c199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.622+0000 7f9f7b7fe640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c1027a0 0x7f9f7c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 -- 172.21.15.107:0/662601612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c103160 msgr2=0x7f9f7c199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c103160 0x7f9f7c199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 -- 172.21.15.107:0/662601612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c1027a0 msgr2=0x7f9f7c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c1027a0 0x7f9f7c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 -- 172.21.15.107:0/662601612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f7c19a640 con 0x7f9f7c101640 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f7bfff640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009a50 tx=0x7f9f6c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c03f030 con 0x7f9f7c101640 2024-08-20T06:32:58.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9f6c003a80 con 0x7f9f7c101640 2024-08-20T06:32:58.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f7c19a8d0 con 0x7f9f7c101640 2024-08-20T06:32:58.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c035070 con 0x7f9f7c101640 2024-08-20T06:32:58.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.623+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f7c1a0ec0 con 0x7f9f7c101640 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.624+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f7c108a90 con 0x7f9f7c101640 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.625+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f9f6c042030 con 0x7f9f7c101640 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.626+0000 7f9f797fa640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 0x7f9f640b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.626+0000 7f9f7b7fe640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 0x7f9f640b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.626+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5549+0+0 (secure 0 0 0) 0x7f9f6c103af0 con 0x7f9f7c101640 2024-08-20T06:32:58.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.626+0000 7f9f7b7fe640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 0x7f9f640b6090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009bd0 tx=0x7f9f70005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:32:58.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.629+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f9f6c0cb520 con 0x7f9f7c101640 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: osdmap e48: 9 total, 8 up, 9 in 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:32:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:32:58.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:58 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:32:58.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:32:58.814+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f9f7c0632c0 con 0x7f9f640b3bd0 2024-08-20T06:32:59.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:32:59.144+0000 7f3221415640 -1 osd.8 0 waiting for initial osdmap 2024-08-20T06:32:59.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:32:59.390+0000 7f3219234640 -1 osd.8 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi193 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi193 to 5696M 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:59.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:59.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:59.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:59.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='osd.8 ' entity='osd.8' 2024-08-20T06:32:59.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:32:59 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi193 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi193 to 5696M 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:59.558 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:59.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:59.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:59.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='osd.8 ' entity='osd.8' 2024-08-20T06:32:59.559 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:32:59 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:32:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi193 2024-08-20T06:32:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi193 to 5696M 2024-08-20T06:32:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:32:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: osdmap e49: 9 total, 8 up, 9 in 2024-08-20T06:32:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:32:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:32:59.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:32:59.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:32:59.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:32:59.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='osd.8 ' entity='osd.8' 2024-08-20T06:32:59.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:32:59 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.525 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:00 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:00 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: osdmap e50: 9 total, 8 up, 9 in 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:00 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] boot 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: osdmap e51: 9 total, 9 up, 9 in 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"} : dispatch 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"}]': finished 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: osdmap e52: 10 total, 9 up, 10 in 2024-08-20T06:33:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:01 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] boot 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: osdmap e51: 9 total, 9 up, 9 in 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"} : dispatch 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"}]': finished 2024-08-20T06:33:01.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: osdmap e52: 10 total, 9 up, 10 in 2024-08-20T06:33:01.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:01 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: osd.8 [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] boot 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: osdmap e51: 9 total, 9 up, 9 in 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"} : dispatch 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1641170823' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec582cc0-f65e-403c-bd4f-ab7077dd1f38"}]': finished 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: osdmap e52: 10 total, 9 up, 10 in 2024-08-20T06:33:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:01 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:02 smithi183 ceph-mon[22204]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:02 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2971464981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:02.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:02 smithi193 ceph-mon[21944]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:02.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:02 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2971464981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:02 smithi107 ceph-mon[18495]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:02 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2971464981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:03.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:03 smithi183 ceph-mon[22204]: osdmap e53: 10 total, 9 up, 10 in 2024-08-20T06:33:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:03 smithi193 ceph-mon[21944]: osdmap e53: 10 total, 9 up, 10 in 2024-08-20T06:33:03.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:03 smithi107 ceph-mon[18495]: osdmap e53: 10 total, 9 up, 10 in 2024-08-20T06:33:03.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:04 smithi183 ceph-mon[22204]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:04 smithi193 ceph-mon[21944]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:04.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:04 smithi107 ceph-mon[18495]: pgmap v194: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:06 smithi183 ceph-mon[22204]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:06.768 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:06 smithi107 ceph-mon[18495]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:06.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:06 smithi193 ceph-mon[21944]: pgmap v195: 1 pgs: 1 peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:08.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:08 smithi107 ceph-mon[18495]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-08-20T06:33:08.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:08 smithi183 ceph-mon[22204]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-08-20T06:33:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:08 smithi193 ceph-mon[21944]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-08-20T06:33:10.685 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:10 smithi107 ceph-mon[18495]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:10.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T06:33:10.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:10.686 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:10 smithi107 ceph-mon[18495]: Deploying daemon osd.9 on smithi107 2024-08-20T06:33:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:10 smithi183 ceph-mon[22204]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T06:33:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:10 smithi183 ceph-mon[22204]: Deploying daemon osd.9 on smithi107 2024-08-20T06:33:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:10 smithi193 ceph-mon[21944]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T06:33:10.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:10.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:10 smithi193 ceph-mon[21944]: Deploying daemon osd.9 on smithi107 2024-08-20T06:33:12.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:12 smithi107 ceph-mon[18495]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T06:33:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:12 smithi183 ceph-mon[22204]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T06:33:12.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:12 smithi193 ceph-mon[21944]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T06:33:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:14 smithi107 ceph-mon[18495]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:33:14.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:14 smithi183 ceph-mon[22204]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:33:14.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:14 smithi193 ceph-mon[21944]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-08-20T06:33:16.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:16 smithi183 ceph-mon[22204]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:16.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:16 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:16 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:16.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:16 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:16.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:16 smithi193 ceph-mon[21944]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:16.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:16 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:16.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:16 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:16.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:16 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:16 smithi107 ceph-mon[18495]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:16 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:16 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:16 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.398+0000 7f9f797fa640 1 -- 172.21.15.107:0/662601612 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9f7c0632c0 con 0x7f9f640b3bd0 2024-08-20T06:33:18.400 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 9 on host 'smithi107' 2024-08-20T06:33:18.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.400+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 msgr2=0x7f9f640b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:18.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.400+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 0x7f9f640b6090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f70009bd0 tx=0x7f9f70005f90 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.400+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 msgr2=0x7f9f7c198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.400+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009a50 tx=0x7f9f6c004040 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 shutdown_connections 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9f640b3bd0 0x7f9f640b6090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f7c103160 0x7f9f7c199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9f7c1027a0 0x7f9f7c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 --2- 172.21.15.107:0/662601612 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9f7c101640 0x7f9f7c198b10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 >> 172.21.15.107:0/662601612 conn(0x7f9f7c0fcf30 msgr2=0x7f9f7c105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 shutdown_connections 2024-08-20T06:33:18.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:33:18.401+0000 7f9f8242e640 1 -- 172.21.15.107:0/662601612 wait complete. 2024-08-20T06:33:18.725 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:18 smithi107 ceph-mon[18495]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:18.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.726 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:18 smithi183 ceph-mon[22204]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:18.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:18 smithi193 ceph-mon[21944]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:33:18.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:18.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:19.038 DEBUG:teuthology.orchestra.run.smithi107:osd.9> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.9.service 2024-08-20T06:33:19.042 INFO:tasks.cephadm:Deploying osd.10 on smithi183 with /dev/vg_nvme/lv_1... 2024-08-20T06:33:19.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T06:33:19.295 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:33:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:20 smithi107 ceph-mon[18495]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:20.968 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:20 smithi183 ceph-mon[22204]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:20.968 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:20.968 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:21.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:20 smithi193 ceph-mon[21944]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:21.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:21.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:21.761 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-08-20T06:33:21.790 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi183:vg_nvme/lv_1 2024-08-20T06:33:22.038 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:33:22.355 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:33:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:33:21.992+0000 7fdcfdc8d640 -1 osd.9 0 log_to_monitors true 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 -- 172.21.15.183:0/3757419400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 msgr2=0x7f6094109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 --2- 172.21.15.183:0/3757419400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094109af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f608c009cd0 tx=0x7f608c02a8e0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 -- 172.21.15.183:0/3757419400 shutdown_connections 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 --2- 172.21.15.183:0/3757419400 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094109af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 --2- 172.21.15.183:0/3757419400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 0x7f6094102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 --2- 172.21.15.183:0/3757419400 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 0x7f6094101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 -- 172.21.15.183:0/3757419400 >> 172.21.15.183:0/3757419400 conn(0x7f60940fcf00 msgr2=0x7f60940ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 -- 172.21.15.183:0/3757419400 shutdown_connections 2024-08-20T06:33:22.494 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.493+0000 7f609a6a3640 1 -- 172.21.15.183:0/3757419400 wait complete. 2024-08-20T06:33:22.496 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 Processor -- start 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 -- start start 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 0x7f6094197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 0x7f6094197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f609410d160 con 0x7f6094101650 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f609410cfe0 con 0x7f6094103170 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f609a6a3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f609410d2e0 con 0x7f60941027b0 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6098c40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6098c40640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.183:45996/0 (socket says 172.21.15.183:45996) 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6098c40640 1 -- 172.21.15.183:0/2200716759 learned_addr learned my addr 172.21.15.183:0/2200716759 (peer_addr_for_me v2:172.21.15.183:0/0) 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f60937fe640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 0x7f6094197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:22.497 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6093fff640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 0x7f6094197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6098c40640 1 -- 172.21.15.183:0/2200716759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 msgr2=0x7f6094197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.494+0000 7f6098c40640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 0x7f6094197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f6098c40640 1 -- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 msgr2=0x7f6094197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f6098c40640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 0x7f6094197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f6098c40640 1 -- 172.21.15.183:0/2200716759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6094199470 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f6098c40640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f608c006010 tx=0x7f608c003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c004240 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6094199700 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f608c0305a0 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608c03b660 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.495+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609419fe40 con 0x7f6094103170 2024-08-20T06:33:22.498 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.496+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6094101a50 con 0x7f6094103170 2024-08-20T06:33:22.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.497+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f608c0309e0 con 0x7f6094103170 2024-08-20T06:33:22.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.498+0000 7f60917fa640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 0x7f60800b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:22.499 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.498+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5944+0+0 (secure 0 0 0) 0x7f608c02e030 con 0x7f6094103170 2024-08-20T06:33:22.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.498+0000 7f6093fff640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 0x7f60800b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:22.500 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.498+0000 7f6093fff640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 0x7f60800b6090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6084009bb0 tx=0x7f6084005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:33:22.502 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.502+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f608c0cc310 con 0x7f6094103170 2024-08-20T06:33:22.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:22 smithi107 ceph-mon[18495]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:22.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:22 smithi107 ceph-mon[18495]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T06:33:22.688 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:22.686+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f60941a0010 con 0x7f60800b3bd0 2024-08-20T06:33:22.688 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:22 smithi183 ceph-mon[22204]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:22.688 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:22 smithi183 ceph-mon[22204]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T06:33:23.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:22 smithi193 ceph-mon[21944]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:23.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:22 smithi193 ceph-mon[21944]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: osdmap e54: 10 total, 9 up, 10 in 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:23.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi107 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi107 to 4271M 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:23.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:23.988 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: osdmap e54: 10 total, 9 up, 10 in 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi107 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi107 to 4271M 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:24.056 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T06:33:24.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: osdmap e54: 10 total, 9 up, 10 in 2024-08-20T06:33:24.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi183:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi107 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:24.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi107 to 4271M 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:24.113 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.634 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' 2024-08-20T06:33:24.635 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:24 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:33:24.490+0000 7fdcf9c10640 -1 osd.9 0 waiting for initial osdmap 2024-08-20T06:33:24.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:33:24.679+0000 7fdcf1a2f640 -1 osd.9 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:33:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:33:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T06:33:24.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:24.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' 2024-08-20T06:33:24.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:25.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:25.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-08-20T06:33:25.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: osdmap e55: 10 total, 9 up, 10 in 2024-08-20T06:33:25.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:25.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:25.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: from='osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516]' entity='osd.9' 2024-08-20T06:33:25.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:24 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/2466455991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"}]': finished 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] boot 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: osdmap e56: 11 total, 10 up, 11 in 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:25.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:25 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:26.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:33:26.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/2466455991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"}]': finished 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] boot 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: osdmap e56: 11 total, 10 up, 11 in 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:26.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:25 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/2466455991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"} : dispatch 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a94743b-6ba0-4a13-878b-a5e931783b7d"}]': finished 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: osd.9 [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] boot 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: osdmap e56: 11 total, 10 up, 11 in 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T06:33:26.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:25 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:26.993 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:26 smithi183 ceph-mon[22204]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:26.993 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:26 smithi183 ceph-mon[22204]: from='client.? 172.21.15.183:0/1203860397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:27.076 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:26 smithi193 ceph-mon[21944]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:27.076 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:26 smithi193 ceph-mon[21944]: from='client.? 172.21.15.183:0/1203860397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:27.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:26 smithi107 ceph-mon[18495]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-08-20T06:33:27.124 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:26 smithi107 ceph-mon[18495]: from='client.? 172.21.15.183:0/1203860397' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:27 smithi183 ceph-mon[22204]: osdmap e57: 11 total, 10 up, 11 in 2024-08-20T06:33:27.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:27 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:28.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:27 smithi193 ceph-mon[21944]: osdmap e57: 11 total, 10 up, 11 in 2024-08-20T06:33:28.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:27 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:28.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:27 smithi107 ceph-mon[18495]: osdmap e57: 11 total, 10 up, 11 in 2024-08-20T06:33:28.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:27 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:28.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:28 smithi183 ceph-mon[22204]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:28 smithi193 ceph-mon[21944]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:28 smithi107 ceph-mon[18495]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:30 smithi183 ceph-mon[22204]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:31.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:30 smithi193 ceph-mon[21944]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:31.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:30 smithi107 ceph-mon[18495]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:32.860 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:32 smithi183 ceph-mon[22204]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:33.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:32 smithi193 ceph-mon[21944]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:33.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:32 smithi107 ceph-mon[18495]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:33 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T06:33:33.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:33 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:34.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:33 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T06:33:34.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:33 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:34.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:33 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T06:33:34.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:33 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:34.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:34 smithi183 ceph-mon[22204]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:34.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:34 smithi183 ceph-mon[22204]: Deploying daemon osd.10 on smithi183 2024-08-20T06:33:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:34 smithi193 ceph-mon[21944]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:34 smithi193 ceph-mon[21944]: Deploying daemon osd.10 on smithi183 2024-08-20T06:33:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:34 smithi107 ceph-mon[18495]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:34 smithi107 ceph-mon[18495]: Deploying daemon osd.10 on smithi183 2024-08-20T06:33:36.927 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:36 smithi183 ceph-mon[22204]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:36 smithi193 ceph-mon[21944]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:37.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:36 smithi107 ceph-mon[18495]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:38.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:38 smithi183 ceph-mon[22204]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:39.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:38 smithi193 ceph-mon[21944]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:39.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:38 smithi107 ceph-mon[18495]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:39.815 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:39.815 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:39.815 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:40.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:40.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:41.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:40 smithi193 ceph-mon[21944]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:41.067 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:40 smithi183 ceph-mon[22204]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:41.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:40 smithi107 ceph-mon[18495]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:42.012 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.011+0000 7f60917fa640 1 -- 172.21.15.183:0/2200716759 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f60941a0010 con 0x7f60800b3bd0 2024-08-20T06:33:42.013 INFO:teuthology.orchestra.run.smithi183.stdout:Created osd(s) 10 on host 'smithi183' 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 msgr2=0x7f60800b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 0x7f60800b6090 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6084009bb0 tx=0x7f6084005f90 comp rx=0 tx=0).stop 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 msgr2=0x7f6094198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f608c006010 tx=0x7f608c003ba0 comp rx=0 tx=0).stop 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 shutdown_connections 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6094103170 0x7f6094198bd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60800b3bd0 0x7f60800b6090 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60941027b0 0x7f6094197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:42.015 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 --2- 172.21.15.183:0/2200716759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6094101650 0x7f6094197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:42.016 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 >> 172.21.15.183:0/2200716759 conn(0x7f60940fcf00 msgr2=0x7f6094103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:33:42.016 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 shutdown_connections 2024-08-20T06:33:42.016 INFO:teuthology.orchestra.run.smithi183.stderr:2024-08-20T06:33:42.014+0000 7f609a6a3640 1 -- 172.21.15.183:0/2200716759 wait complete. 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:42.680 DEBUG:teuthology.orchestra.run.smithi183:osd.10> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.10.service 2024-08-20T06:33:42.683 INFO:tasks.cephadm:Deploying osd.11 on smithi193 with /dev/vg_nvme/lv_1... 2024-08-20T06:33:42.683 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T06:33:42.940 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:33:43.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:43 smithi193 ceph-mon[21944]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:43 smithi107 ceph-mon[18495]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:43.640 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:43 smithi183 ceph-mon[22204]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:44.938 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:33:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:33:44.630+0000 7f312a06f640 -1 osd.10 0 log_to_monitors true 2024-08-20T06:33:45.445 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-08-20T06:33:45.470 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:45 smithi193 ceph-mon[21944]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:45.470 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:45 smithi193 ceph-mon[21944]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.470 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:45 smithi193 ceph-mon[21944]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.471 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_1 2024-08-20T06:33:45.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:45 smithi183 ceph-mon[22204]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:45 smithi183 ceph-mon[22204]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:45 smithi183 ceph-mon[22204]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:45 smithi107 ceph-mon[18495]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:45 smithi107 ceph-mon[18495]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:45 smithi107 ceph-mon[18495]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T06:33:45.728 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:33:46.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.129+0000 7f327c9d9640 1 -- 172.21.15.193:0/2279654963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 msgr2=0x7f32781066d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:46.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.129+0000 7f327c9d9640 1 --2- 172.21.15.193:0/2279654963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781066d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3270009ca0 tx=0x7f327002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 -- 172.21.15.193:0/2279654963 shutdown_connections 2024-08-20T06:33:46.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 --2- 172.21.15.193:0/2279654963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781066d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.133 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 --2- 172.21.15.193:0/2279654963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3278103670 0x7f3278105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 --2- 172.21.15.193:0/2279654963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f327806b3e0 0x7f32781030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 -- 172.21.15.193:0/2279654963 >> 172.21.15.193:0/2279654963 conn(0x7f32780fcee0 msgr2=0x7f32780ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 -- 172.21.15.193:0/2279654963 shutdown_connections 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 -- 172.21.15.193:0/2279654963 wait complete. 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 Processor -- start 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.130+0000 7f327c9d9640 1 -- start start 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 0x7f327819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3278114860 con 0x7f327806b3e0 2024-08-20T06:33:46.134 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32781146e0 con 0x7f3278103670 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f32781149e0 con 0x7f3278106220 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 0x7f327819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3275d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.193:50552/0 (socket says 172.21.15.193:50552) 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 0x7f327819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.193:35914/0 (socket says 172.21.15.193:35914) 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3275d9b640 1 -- 172.21.15.193:0/1208557560 learned_addr learned my addr 172.21.15.193:0/1208557560 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:56014/0 (socket says 172.21.15.193:56014) 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 -- 172.21.15.193:0/1208557560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 msgr2=0x7f327819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 -- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 msgr2=0x7f327819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 0x7f327819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:33:46.135 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 -- 172.21.15.193:0/1208557560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32781a0a90 con 0x7f3278106220 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3275d9b640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f3276d9d640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3270031040 tx=0x7f3270003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270004040 con 0x7f3278106220 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.131+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32781a0d20 con 0x7f3278106220 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.132+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3270004670 con 0x7f3278106220 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.132+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270030cc0 con 0x7f3278106220 2024-08-20T06:33:46.136 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.132+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32781a7460 con 0x7f3278106220 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.133+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32781066d0 con 0x7f3278106220 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.134+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f327004e440 con 0x7f3278106220 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.135+0000 7f32677fe640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.135+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6398+0+0 (secure 0 0 0) 0x7f32701048b0 con 0x7f3278106220 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.135+0000 7f327659c640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:33:46.137 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.136+0000 7f327659c640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f326c00bbe0 tx=0x7f326c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:33:46.140 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.139+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f32700cc680 con 0x7f3278106220 2024-08-20T06:33:46.339 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:33:46.338+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f32780632c0 con 0x7f32600b3bd0 2024-08-20T06:33:46.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi183 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: osdmap e58: 11 total, 10 up, 11 in 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:46.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:46.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:46 smithi193 ceph-mon[21944]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi183 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: osdmap e58: 11 total, 10 up, 11 in 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:46 smithi107 ceph-mon[18495]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:33:46.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:33:46.262+0000 7f3125ff2640 -1 osd.10 0 waiting for initial osdmap 2024-08-20T06:33:46.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:33:46.276+0000 7f3121e19640 -1 osd.10 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:33:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T06:33:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi183 2024-08-20T06:33:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: osdmap e58: 11 total, 10 up, 11 in 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:33:46.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:33:46.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:46 smithi183 ceph-mon[22204]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi183", "root=default"]}]': finished 2024-08-20T06:33:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi183 to 4272M 2024-08-20T06:33:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: osdmap e59: 11 total, 10 up, 11 in 2024-08-20T06:33:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] boot 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: osdmap e60: 11 total, 11 up, 11 in 2024-08-20T06:33:47.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:47 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi183 to 4272M 2024-08-20T06:33:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: osdmap e59: 11 total, 10 up, 11 in 2024-08-20T06:33:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] boot 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: osdmap e60: 11 total, 11 up, 11 in 2024-08-20T06:33:47.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:47 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi183 to 4272M 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: osdmap e59: 11 total, 10 up, 11 in 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: osd.10 [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] boot 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: osdmap e60: 11 total, 11 up, 11 in 2024-08-20T06:33:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:47 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T06:33:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:48 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:33:48.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:48 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:33:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:48 smithi193 ceph-mon[21944]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:48 smithi193 ceph-mon[21944]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:48.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:48 smithi193 ceph-mon[21944]: osdmap e61: 11 total, 11 up, 11 in 2024-08-20T06:33:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:48 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:33:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:48 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:33:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:48 smithi107 ceph-mon[18495]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:48 smithi107 ceph-mon[18495]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:48.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:48 smithi107 ceph-mon[18495]: osdmap e61: 11 total, 11 up, 11 in 2024-08-20T06:33:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:48 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:33:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:48 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:33:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:48 smithi183 ceph-mon[22204]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:33:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:48 smithi183 ceph-mon[22204]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:48.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:48 smithi183 ceph-mon[22204]: osdmap e61: 11 total, 11 up, 11 in 2024-08-20T06:33:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/903994967' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"}]': finished 2024-08-20T06:33:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: osdmap e62: 12 total, 11 up, 12 in 2024-08-20T06:33:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:33:49.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:49 smithi193 ceph-mon[21944]: from='client.? 172.21.15.193:0/3083987352' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/903994967' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"}]': finished 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: osdmap e62: 12 total, 11 up, 12 in 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:33:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:49 smithi107 ceph-mon[18495]: from='client.? 172.21.15.193:0/3083987352' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/903994967' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"} : dispatch 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e25cf09b-b923-401d-9d8b-16f2cfff57bd"}]': finished 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: osdmap e62: 12 total, 11 up, 12 in 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:33:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:49 smithi183 ceph-mon[22204]: from='client.? 172.21.15.193:0/3083987352' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T06:33:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:50 smithi193 ceph-mon[21944]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:50.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:50 smithi107 ceph-mon[18495]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:50 smithi183 ceph-mon[22204]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:52 smithi107 ceph-mon[18495]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:52.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:52 smithi183 ceph-mon[22204]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:52 smithi193 ceph-mon[21944]: pgmap v227: 1 pgs: 1 remapped+peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:33:54.520 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:54 smithi193 ceph-mon[21944]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:33:54.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:54 smithi107 ceph-mon[18495]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:33:54.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:54 smithi183 ceph-mon[22204]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-08-20T06:33:56.540 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:56 smithi193 ceph-mon[21944]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:56.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:56 smithi107 ceph-mon[18495]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:56 smithi183 ceph-mon[22204]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T06:33:57.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T06:33:57.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:57.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:57 smithi107 ceph-mon[18495]: Deploying daemon osd.11 on smithi193 2024-08-20T06:33:57.653 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T06:33:57.653 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:57.653 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:57 smithi193 ceph-mon[21944]: Deploying daemon osd.11 on smithi193 2024-08-20T06:33:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T06:33:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:33:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:57 smithi183 ceph-mon[22204]: Deploying daemon osd.11 on smithi193 2024-08-20T06:33:58.584 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:33:58 smithi193 ceph-mon[21944]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:33:58.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:33:58 smithi107 ceph-mon[18495]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:33:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:33:58 smithi183 ceph-mon[22204]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T06:34:00.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:00 smithi107 ceph-mon[18495]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T06:34:00.679 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:00 smithi193 ceph-mon[21944]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T06:34:00.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:00 smithi183 ceph-mon[22204]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-08-20T06:34:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:02 smithi193 ceph-mon[21944]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:02 smithi107 ceph-mon[18495]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:02 smithi183 ceph-mon[22204]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:04.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:04 smithi107 ceph-mon[18495]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:04.588 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:04 smithi193 ceph-mon[21944]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:04 smithi183 ceph-mon[22204]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T06:34:05.628 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 11 on host 'smithi193' 2024-08-20T06:34:05.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.627+0000 7f32677fe640 1 -- 172.21.15.193:0/1208557560 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f32780632c0 con 0x7f32600b3bd0 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 msgr2=0x7f32600b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f326c00bbe0 tx=0x7f326c00b390 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 msgr2=0x7f32781a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3270031040 tx=0x7f3270003ba0 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 shutdown_connections 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278106220 0x7f32781a01f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278103670 0x7f327819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 --2- 172.21.15.193:0/1208557560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f327806b3e0 0x7f327819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 >> 172.21.15.193:0/1208557560 conn(0x7f32780fcee0 msgr2=0x7f3278101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:05.631 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.629+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 shutdown_connections 2024-08-20T06:34:05.632 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:05.630+0000 7f327c9d9640 1 -- 172.21.15.193:0/1208557560 wait complete. 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:05 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.275 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.276 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:05 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.305 DEBUG:teuthology.orchestra.run.smithi193:osd.11> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.11.service 2024-08-20T06:34:06.307 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-20T06:34:06.307 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T06:34:06.335 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.335 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.335 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:06.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:06.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:06.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.336 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:05 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:06.562 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:06.941 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:06 smithi107 ceph-mon[18495]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 -- 172.21.15.107:0/2129256120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c06bdc0 msgr2=0x7f492c105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 --2- 172.21.15.107:0/2129256120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c06bdc0 0x7f492c105330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4924009ca0 tx=0x7f492402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 -- 172.21.15.107:0/2129256120 shutdown_connections 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 --2- 172.21.15.107:0/2129256120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c108470 0x7f492c10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 --2- 172.21.15.107:0/2129256120 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1058c0 0x7f492c107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 --2- 172.21.15.107:0/2129256120 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c06bdc0 0x7f492c105330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 -- 172.21.15.107:0/2129256120 >> 172.21.15.107:0/2129256120 conn(0x7f492c0fcf00 msgr2=0x7f492c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 -- 172.21.15.107:0/2129256120 shutdown_connections 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 -- 172.21.15.107:0/2129256120 wait complete. 2024-08-20T06:34:07.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.021+0000 7f4931547640 1 Processor -- start 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- start start 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c06bdc0 0x7f492c102a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1058c0 0x7f492c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10e2f0 con 0x7f492c108470 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10e170 con 0x7f492c06bdc0 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10e470 con 0x7f492c1058c0 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58412/0 (socket says 172.21.15.107:58412) 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 -- 172.21.15.107:0/3561458842 learned_addr learned my addr 172.21.15.107:0/3561458842 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492a7fc640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1058c0 0x7f492c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:07.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492affd640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c06bdc0 0x7f492c102a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 -- 172.21.15.107:0/3561458842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1058c0 msgr2=0x7f492c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1058c0 0x7f492c1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 -- 172.21.15.107:0/3561458842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c06bdc0 msgr2=0x7f492c102a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c06bdc0 0x7f492c102a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 -- 172.21.15.107:0/3561458842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c1020d0 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f492b7fe640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e970 tx=0x7f491c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c00ccc0 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c1a2bb0 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f491c00ce60 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c1a3110 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.022+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c010430 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.023+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c0fe9f0 con 0x7f492c108470 2024-08-20T06:34:07.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.026+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f491c004ef0 con 0x7f492c108470 2024-08-20T06:34:07.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.026+0000 7f4923fff640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:07.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.026+0000 7f492affd640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:07.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.026+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6809+0+0 (secure 0 0 0) 0x7f491c0e3b70 con 0x7f492c108470 2024-08-20T06:34:07.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.027+0000 7f492affd640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4924000f80 tx=0x7f4924031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:07.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.029+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f491c055e20 con 0x7f492c108470 2024-08-20T06:34:07.211 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.210+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f492c02e4f0 con 0x7f492c108470 2024-08-20T06:34:07.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.211+0000 7f4923fff640 1 -- 172.21.15.107:0/3561458842 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v62) ==== 74+0+133 (secure 0 0 0) 0x7f491c0ab7a0 con 0x7f492c108470 2024-08-20T06:34:07.212 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 msgr2=0x7f49180b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4924000f80 tx=0x7f4924031000 comp rx=0 tx=0).stop 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 msgr2=0x7f492c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e970 tx=0x7f491c00ed70 comp rx=0 tx=0).stop 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 shutdown_connections 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c108470 0x7f492c101810 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1058c0 0x7f492c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 --2- 172.21.15.107:0/3561458842 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c06bdc0 0x7f492c102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:07.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 >> 172.21.15.107:0/3561458842 conn(0x7f492c0fcf00 msgr2=0x7f492c103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:07.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 shutdown_connections 2024-08-20T06:34:07.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:07.213+0000 7f4931547640 1 -- 172.21.15.107:0/3561458842 wait complete. 2024-08-20T06:34:07.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:06 smithi183 ceph-mon[22204]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:06 smithi193 ceph-mon[21944]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:08.015 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":62,"num_osds":12,"num_up_osds":11,"osd_up_since":1724135626,"num_in_osds":12,"osd_in_since":1724135628,"num_remapped_pgs":0} 2024-08-20T06:34:08.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:07 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3561458842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:07 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3561458842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:08.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:07 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3561458842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:08.903 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:34:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:34:08.634+0000 7f78f6bbf640 -1 osd.11 0 log_to_monitors true 2024-08-20T06:34:09.016 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T06:34:09.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:08 smithi183 ceph-mon[22204]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:08 smithi183 ceph-mon[22204]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:08 smithi183 ceph-mon[22204]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:08 smithi193 ceph-mon[21944]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:08 smithi193 ceph-mon[21944]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:08 smithi193 ceph-mon[21944]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:08 smithi107 ceph-mon[18495]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:09.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:08 smithi107 ceph-mon[18495]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:08 smithi107 ceph-mon[18495]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T06:34:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 -- 172.21.15.107:0/2654632271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460102770 msgr2=0x7f8460102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 --2- 172.21.15.107:0/2654632271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460102770 0x7f8460102bf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8450009930 tx=0x7f845002a420 comp rx=0 tx=0).stop 2024-08-20T06:34:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 -- 172.21.15.107:0/2654632271 shutdown_connections 2024-08-20T06:34:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 --2- 172.21.15.107:0/2654632271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f8460109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 --2- 172.21.15.107:0/2654632271 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460102770 0x7f8460102bf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 --2- 172.21.15.107:0/2654632271 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460101610 0x7f8460101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 -- 172.21.15.107:0/2654632271 >> 172.21.15.107:0/2654632271 conn(0x7f84600fcee0 msgr2=0x7f84600ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 -- 172.21.15.107:0/2654632271 shutdown_connections 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.710+0000 7f8467961640 1 -- 172.21.15.107:0/2654632271 wait complete. 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 Processor -- start 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 -- start start 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460102770 0x7f846019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f846019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f846010d140 con 0x7f8460101610 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f846010cfc0 con 0x7f8460102770 2024-08-20T06:34:09.733 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8467961640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f846010d2c0 con 0x7f8460103130 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f84656fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f84656fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58424/0 (socket says 172.21.15.107:58424) 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f84656fd640 1 -- 172.21.15.107:0/2371268474 learned_addr learned my addr 172.21.15.107:0/2371268474 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.711+0000 7f8465efe640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f846019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f8464efc640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460102770 0x7f846019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 -- 172.21.15.107:0/2371268474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 msgr2=0x7f846019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f846019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 -- 172.21.15.107:0/2371268474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460102770 msgr2=0x7f846019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460102770 0x7f846019d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 -- 172.21.15.107:0/2371268474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84601a05c0 con 0x7f8460101610 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f8465efe640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f846019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:34:09.734 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84656fd640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d950 tx=0x7f845c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c017030 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84601a08b0 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f845c004020 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c0055d0 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.712+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84601a0e40 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.713+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84601a10d0 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.714+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f845c009a30 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.715+0000 7f84567fc640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 0x7f844c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.715+0000 7f8464efc640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 0x7f844c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.715+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 6831+0+0 (secure 0 0 0) 0x7f845c0e3800 con 0x7f8460101610 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.715+0000 7f8464efc640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 0x7f844c0b6090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f84500096c0 tx=0x7f8450000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:09.735 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.718+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f845c0ab420 con 0x7f8460101610 2024-08-20T06:34:09.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.895+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f84601a12a0 con 0x7f8460101610 2024-08-20T06:34:09.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.895+0000 7f84567fc640 1 -- 172.21.15.107:0/2371268474 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v63) ==== 74+0+133 (secure 0 0 0) 0x7f845c0aad20 con 0x7f8460101610 2024-08-20T06:34:09.896 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 msgr2=0x7f844c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 0x7f844c0b6090 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f84500096c0 tx=0x7f8450000bc0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 msgr2=0x7f846006b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f845c00d950 tx=0x7f845c00de10 comp rx=0 tx=0).stop 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 shutdown_connections 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f844c0b3bd0 0x7f844c0b6090 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8460103130 0x7f846019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8460102770 0x7f846019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 --2- 172.21.15.107:0/2371268474 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8460101610 0x7f846006b0f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 >> 172.21.15.107:0/2371268474 conn(0x7f84600fcee0 msgr2=0x7f8460103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 shutdown_connections 2024-08-20T06:34:09.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:09.897+0000 7f8467961640 1 -- 172.21.15.107:0/2371268474 wait complete. 2024-08-20T06:34:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T06:34:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: osdmap e63: 12 total, 11 up, 12 in 2024-08-20T06:34:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: Detected new or changed devices on smithi193 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi193 to 4272M 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:10.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.238 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:09 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2371268474' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:10.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:34:09.915+0000 7f78f2b42640 -1 osd.11 0 waiting for initial osdmap 2024-08-20T06:34:10.305 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:34:09.928+0000 7f78ee969640 -1 osd.11 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: osdmap e63: 12 total, 11 up, 12 in 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: Detected new or changed devices on smithi193 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi193 to 4272M 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:09 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2371268474' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T06:34:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: osdmap e63: 12 total, 11 up, 12 in 2024-08-20T06:34:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: Detected new or changed devices on smithi193 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi193 to 4272M 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:10.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:10.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:09 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2371268474' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:10.700 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":63,"num_osds":12,"num_up_osds":11,"osd_up_since":1724135626,"num_in_osds":12,"osd_in_since":1724135628,"num_remapped_pgs":0} 2024-08-20T06:34:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:10 smithi183 ceph-mon[22204]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:10 smithi183 ceph-mon[22204]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:34:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:10 smithi183 ceph-mon[22204]: osdmap e64: 12 total, 11 up, 12 in 2024-08-20T06:34:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:10 smithi193 ceph-mon[21944]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:10 smithi193 ceph-mon[21944]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:34:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:10 smithi193 ceph-mon[21944]: osdmap e64: 12 total, 11 up, 12 in 2024-08-20T06:34:11.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:10 smithi107 ceph-mon[18495]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:10 smithi107 ceph-mon[18495]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-08-20T06:34:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:10 smithi107 ceph-mon[18495]: osdmap e64: 12 total, 11 up, 12 in 2024-08-20T06:34:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:11.701 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T06:34:11.959 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:12.080 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:11 smithi107 ceph-mon[18495]: purged_snaps scrub starts 2024-08-20T06:34:12.080 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:11 smithi107 ceph-mon[18495]: purged_snaps scrub ok 2024-08-20T06:34:12.081 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:11 smithi107 ceph-mon[18495]: osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] boot 2024-08-20T06:34:12.081 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:11 smithi107 ceph-mon[18495]: osdmap e65: 12 total, 12 up, 12 in 2024-08-20T06:34:12.081 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:12.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:11 smithi183 ceph-mon[22204]: purged_snaps scrub starts 2024-08-20T06:34:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:11 smithi183 ceph-mon[22204]: purged_snaps scrub ok 2024-08-20T06:34:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:11 smithi183 ceph-mon[22204]: osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] boot 2024-08-20T06:34:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:11 smithi183 ceph-mon[22204]: osdmap e65: 12 total, 12 up, 12 in 2024-08-20T06:34:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:11 smithi193 ceph-mon[21944]: purged_snaps scrub starts 2024-08-20T06:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:11 smithi193 ceph-mon[21944]: purged_snaps scrub ok 2024-08-20T06:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:11 smithi193 ceph-mon[21944]: osd.11 [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] boot 2024-08-20T06:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:11 smithi193 ceph-mon[21944]: osdmap e65: 12 total, 12 up, 12 in 2024-08-20T06:34:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 -- 172.21.15.107:0/1525151439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81014b0 msgr2=0x7f1ee81018b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 --2- 172.21.15.107:0/1525151439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81014b0 0x7f1ee81018b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009930 tx=0x7f1ed002a420 comp rx=0 tx=0).stop 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 -- 172.21.15.107:0/1525151439 shutdown_connections 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 --2- 172.21.15.107:0/1525151439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee8102fd0 0x7f1ee8109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 --2- 172.21.15.107:0/1525151439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102610 0x7f1ee8102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 --2- 172.21.15.107:0/1525151439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee81014b0 0x7f1ee81018b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 -- 172.21.15.107:0/1525151439 >> 172.21.15.107:0/1525151439 conn(0x7f1ee80fcd80 msgr2=0x7f1ee80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 -- 172.21.15.107:0/1525151439 shutdown_connections 2024-08-20T06:34:12.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.441+0000 7f1eed002640 1 -- 172.21.15.107:0/1525151439 wait complete. 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.442+0000 7f1eed002640 1 Processor -- start 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.442+0000 7f1eed002640 1 -- start start 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee81014b0 0x7f1ee8105270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102fd0 0x7f1ee8104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810cfc0 con 0x7f1ee8102610 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810ce40 con 0x7f1ee8102fd0 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee810d140 con 0x7f1ee81014b0 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58448/0 (socket says 172.21.15.107:58448) 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 -- 172.21.15.107:0/4092589183 learned_addr learned my addr 172.21.15.107:0/4092589183 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee6d9d640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee81014b0 0x7f1ee8105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:12.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee759e640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102fd0 0x7f1ee8104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 -- 172.21.15.107:0/4092589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee81014b0 msgr2=0x7f1ee8105270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee81014b0 0x7f1ee8105270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 -- 172.21.15.107:0/4092589183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102fd0 msgr2=0x7f1ee8104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102fd0 0x7f1ee8104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 -- 172.21.15.107:0/4092589183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee81048e0 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1ee659c640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c890 tx=0x7f1ed800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8004220 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81a28c0 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1ed8007e40 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed8004a00 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.443+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee81a2dd0 con 0x7f1ee8102610 2024-08-20T06:34:12.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.444+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee81a3090 con 0x7f1ee8102610 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.446+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f1ed80079b0 con 0x7f1ee8102610 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.446+0000 7f1edffff640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 0x7f1ed40b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.446+0000 7f1ee6d9d640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 0x7f1ed40b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.446+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f1ed80e3550 con 0x7f1ee8102610 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.447+0000 7f1ee6d9d640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 0x7f1ed40b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:12.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.450+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1ed8014030 con 0x7f1ee8102610 2024-08-20T06:34:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.626+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f1ee81a3760 con 0x7f1ee8102610 2024-08-20T06:34:12.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.626+0000 7f1edffff640 1 -- 172.21.15.107:0/4092589183 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f1ed80ab040 con 0x7f1ee8102610 2024-08-20T06:34:12.628 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:12.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.628+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 msgr2=0x7f1ed40b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.628+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 0x7f1ed40b6130 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0000c00 tx=0x7f1ed0005720 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.628+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 msgr2=0x7f1ee8103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.628+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ed800c890 tx=0x7f1ed800cd50 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 shutdown_connections 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1ee8102fd0 0x7f1ee8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1ee8102610 0x7f1ee8103ae0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1ed40b3c70 0x7f1ed40b6130 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 --2- 172.21.15.107:0/4092589183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee81014b0 0x7f1ee8105270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 >> 172.21.15.107:0/4092589183 conn(0x7f1ee80fcd80 msgr2=0x7f1ee80fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 shutdown_connections 2024-08-20T06:34:12.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:12.629+0000 7f1eed002640 1 -- 172.21.15.107:0/4092589183 wait complete. 2024-08-20T06:34:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:12 smithi183 ceph-mon[22204]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:12 smithi183 ceph-mon[22204]: osdmap e66: 12 total, 12 up, 12 in 2024-08-20T06:34:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:12 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4092589183' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:12 smithi193 ceph-mon[21944]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:12 smithi193 ceph-mon[21944]: osdmap e66: 12 total, 12 up, 12 in 2024-08-20T06:34:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:12 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4092589183' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:13.309 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:12 smithi107 ceph-mon[18495]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-08-20T06:34:13.310 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:12 smithi107 ceph-mon[18495]: osdmap e66: 12 total, 12 up, 12 in 2024-08-20T06:34:13.310 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:12 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4092589183' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T06:34:13.338 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":66,"num_osds":12,"num_up_osds":12,"osd_up_since":1724135650,"num_in_osds":12,"osd_in_since":1724135628,"num_remapped_pgs":0} 2024-08-20T06:34:13.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:34:13.579 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.000+0000 7f60a479d640 1 -- 172.21.15.107:0/1104360235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c104230 msgr2=0x7f609c10c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.000+0000 7f60a479d640 1 --2- 172.21.15.107:0/1104360235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c104230 0x7f609c10c810 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c02a410 comp rx=0 tx=0).stop 2024-08-20T06:34:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 -- 172.21.15.107:0/1104360235 shutdown_connections 2024-08-20T06:34:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 --2- 172.21.15.107:0/1104360235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c10cde0 0x7f609c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 --2- 172.21.15.107:0/1104360235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c104230 0x7f609c10c810 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 --2- 172.21.15.107:0/1104360235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 0x7f609c103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 -- 172.21.15.107:0/1104360235 >> 172.21.15.107:0/1104360235 conn(0x7f609c0fd400 msgr2=0x7f609c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 -- 172.21.15.107:0/1104360235 shutdown_connections 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 -- 172.21.15.107:0/1104360235 wait complete. 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 Processor -- start 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.001+0000 7f60a479d640 1 -- start start 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 0x7f609c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c104230 0x7f609c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f609c112950 con 0x7f609c10cde0 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1127d0 con 0x7f609c103830 2024-08-20T06:34:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f609c112ad0 con 0x7f609c104230 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58472/0 (socket says 172.21.15.107:58472) 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 -- 172.21.15.107:0/4122613802 learned_addr learned my addr 172.21.15.107:0/4122613802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2539640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 0x7f609c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a1d38640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c104230 0x7f609c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 -- 172.21.15.107:0/4122613802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c104230 msgr2=0x7f609c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c104230 0x7f609c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 -- 172.21.15.107:0/4122613802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 msgr2=0x7f609c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 0x7f609c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 -- 172.21.15.107:0/4122613802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c19efd0 con 0x7f609c10cde0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a2d3a640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f609400cce0 tx=0x7f609400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f609400f450 con 0x7f609c10cde0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f609400fa80 con 0x7f609c10cde0 2024-08-20T06:34:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c19f2c0 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60940189c0 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.002+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a5980 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.003+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f609c103c10 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.005+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f609400fc30 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.005+0000 7f60937fe640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 0x7f60880b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.006+0000 7f60a2539640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 0x7f60880b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.006+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f60940e3cb0 con 0x7f609c10cde0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.006+0000 7f60a2539640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 0x7f60880b6090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6098009990 tx=0x7f6098009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.008+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6094014030 con 0x7f609c10cde0 2024-08-20T06:34:14.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.185+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f609c02e340 con 0x7f609c10cde0 2024-08-20T06:34:14.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.186+0000 7f60937fe640 1 -- 172.21.15.107:0/4122613802 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+18560 (secure 0 0 0) 0x7f60940ab7a0 con 0x7f609c10cde0 2024-08-20T06:34:14.187 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:14.188 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":66,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:34:11.921885+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.188+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 msgr2=0x7f60880b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.188+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 0x7f60880b6090 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6098009990 tx=0x7f6098009110 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 msgr2=0x7f609c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f609400cce0 tx=0x7f609400a620 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 shutdown_connections 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f609c10cde0 0x7f609c19e800 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f60880b3bd0 0x7f60880b6090 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f609c104230 0x7f609c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 --2- 172.21.15.107:0/4122613802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f609c103830 0x7f609c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 >> 172.21.15.107:0/4122613802 conn(0x7f609c0fd400 msgr2=0x7f609c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:14.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 shutdown_connections 2024-08-20T06:34:14.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:14.189+0000 7f60a479d640 1 -- 172.21.15.107:0/4122613802 wait complete. 2024-08-20T06:34:14.861 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T06:30:39.511744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T06:34:14.862 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-20T06:34:15.106 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:15.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:14 smithi107 ceph-mon[18495]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:15.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:14 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4122613802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:34:15.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:14 smithi183 ceph-mon[22204]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:15.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:14 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4122613802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:34:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:14 smithi193 ceph-mon[21944]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:14 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4122613802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 -- 172.21.15.107:0/3776117476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 msgr2=0x7f21b0111510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3776117476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b0111510 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009ca0 tx=0x7f21a802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 -- 172.21.15.107:0/3776117476 shutdown_connections 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3776117476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b0111510 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3776117476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101e90 0x7f21b010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3776117476 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101570 0x7f21b0101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 -- 172.21.15.107:0/3776117476 >> 172.21.15.107:0/3776117476 conn(0x7f21b00fd440 msgr2=0x7f21b00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 -- 172.21.15.107:0/3776117476 shutdown_connections 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.535+0000 7f21b78f2640 1 -- 172.21.15.107:0/3776117476 wait complete. 2024-08-20T06:34:15.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 Processor -- start 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 -- start start 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101e90 0x7f21b019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0114b60 con 0x7f21b010f120 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f21b01149e0 con 0x7f21b0101e90 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b78f2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f21b0114ce0 con 0x7f21b0101570 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b5e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b568e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b5e8f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43106/0 (socket says 172.21.15.107:43106) 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b568e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:47402/0 (socket says 172.21.15.107:47402) 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b5e8f640 1 -- 172.21.15.107:0/3838878509 learned_addr learned my addr 172.21.15.107:0/3838878509 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:15.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.536+0000 7f21b4e8d640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101e90 0x7f21b019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 -- 172.21.15.107:0/3838878509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101e90 msgr2=0x7f21b019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101e90 0x7f21b019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 -- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 msgr2=0x7f21b01a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 -- 172.21.15.107:0/3838878509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b01a11e0 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b5e8f640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b568e640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00d6d0 tx=0x7f21ac00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac017070 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b01a14d0 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f21ac009820 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21ac004bc0 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.537+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b01a7d60 con 0x7f21b0101570 2024-08-20T06:34:15.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.538+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b0101950 con 0x7f21b0101570 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.539+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f21ac005220 con 0x7f21b0101570 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.540+0000 7f21a27fc640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 0x7f219c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.540+0000 7f21b4e8d640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 0x7f219c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.540+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f21ac0e3670 con 0x7f21b0101570 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.540+0000 7f21b4e8d640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 0x7f219c0b6090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21b0111770 tx=0x7f21a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:15.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.543+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f21ac0aaa70 con 0x7f21b0101570 2024-08-20T06:34:15.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.718+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21b010a2e0 con 0x7f21b0101570 2024-08-20T06:34:15.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.718+0000 7f21a27fc640 1 -- 172.21.15.107:0/3838878509 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f21ac0aefc0 con 0x7f21b0101570 2024-08-20T06:34:15.719 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-08-20T06:34:15.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 msgr2=0x7f219c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:15.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 0x7f219c0b6090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f21b0111770 tx=0x7f21a400b390 comp rx=0 tx=0).stop 2024-08-20T06:34:15.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 msgr2=0x7f21b019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:15.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f21ac00d6d0 tx=0x7f21ac00db90 comp rx=0 tx=0).stop 2024-08-20T06:34:15.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 shutdown_connections 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f21b010f120 0x7f21b01a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f21b0101e90 0x7f21b019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f219c0b3bd0 0x7f219c0b6090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 --2- 172.21.15.107:0/3838878509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21b0101570 0x7f21b019b130 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 >> 172.21.15.107:0/3838878509 conn(0x7f21b00fd440 msgr2=0x7f21b00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 shutdown_connections 2024-08-20T06:34:15.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:15.720+0000 7f21b78f2640 1 -- 172.21.15.107:0/3838878509 wait complete. 2024-08-20T06:34:16.212 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:15 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3838878509' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:34:16.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:15 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3838878509' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:34:16.295 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:15 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3838878509' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:34:16.378 INFO:tasks.cephadm:Adding mds.a on smithi107 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.d on smithi107 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.g on smithi107 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.j on smithi107 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.b on smithi183 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.e on smithi183 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.h on smithi183 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.k on smithi183 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.c on smithi193 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.f on smithi193 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.i on smithi193 2024-08-20T06:34:16.379 INFO:tasks.cephadm:Adding mds.l on smithi193 2024-08-20T06:34:16.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l' 2024-08-20T06:34:16.627 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:34:17.026 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:16 smithi193 ceph-mon[21944]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- 172.21.15.193:0/321267387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838104240 msgr2=0x7f083810abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 --2- 172.21.15.193:0/321267387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838104240 0x7f083810abc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0830009d00 tx=0x7f083002a910 comp rx=0 tx=0).stop 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- 172.21.15.193:0/321267387 shutdown_connections 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 --2- 172.21.15.193:0/321267387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838104240 0x7f083810abc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 --2- 172.21.15.193:0/321267387 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838103880 0x7f0838103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 --2- 172.21.15.193:0/321267387 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838102720 0x7f0838102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.151 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- 172.21.15.193:0/321267387 >> 172.21.15.193:0/321267387 conn(0x7f08380fdff0 msgr2=0x7f0838100430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- 172.21.15.193:0/321267387 shutdown_connections 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- 172.21.15.193:0/321267387 wait complete. 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 Processor -- start 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.115+0000 7f083db1e640 1 -- start start 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838102720 0x7f0838198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 0x7f0838199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f083810e2b0 con 0x7f0838104240 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f083810e130 con 0x7f0838102720 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f083810e430 con 0x7f0838103880 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:44766/0 (socket says 172.21.15.193:44766) 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 -- 172.21.15.193:0/1951715202 learned_addr learned my addr 172.21.15.193:0/1951715202 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-08-20T06:34:17.152 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f0837fff640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 0x7f0838199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f08377fe640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838102720 0x7f0838198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 -- 172.21.15.193:0/1951715202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838102720 msgr2=0x7f0838198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838102720 0x7f0838198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 -- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 msgr2=0x7f0838199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 0x7f0838199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 -- 172.21.15.193:0/1951715202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083819a590 con 0x7f0838103880 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f0837fff640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 0x7f0838199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f082effd640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f082000d950 tx=0x7f082000de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0820017030 con 0x7f0838103880 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083819a880 con 0x7f0838103880 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0820004020 con 0x7f0838103880 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.116+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08200055d0 con 0x7f0838103880 2024-08-20T06:34:17.153 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.117+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08381a0f60 con 0x7f0838103880 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.118+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0838109aa0 con 0x7f0838103880 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.119+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f0820009a30 con 0x7f0838103880 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.120+0000 7f08357fa640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 0x7f08240b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.120+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f08200e38e0 con 0x7f0838103880 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.120+0000 7f08377fe640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 0x7f08240b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.120+0000 7f08377fe640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 0x7f08240b6090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f082800a810 tx=0x7f0828008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:17.154 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.123+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f08200ab3d0 con 0x7f0838103880 2024-08-20T06:34:17.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:16 smithi183 ceph-mon[22204]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:17.309 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.307+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}) -- 0x7f08380633c0 con 0x7f08240b3bd0 2024-08-20T06:34:17.318 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.317+0000 7f08357fa640 1 -- 172.21.15.193:0/1951715202 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f08380633c0 con 0x7f08240b3bd0 2024-08-20T06:34:17.319 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mds.all update... 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 msgr2=0x7f08240b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 0x7f08240b6090 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f082800a810 tx=0x7f0828008040 comp rx=0 tx=0).stop 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 msgr2=0x7f08381990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f082000d950 tx=0x7f082000de10 comp rx=0 tx=0).stop 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 shutdown_connections 2024-08-20T06:34:17.321 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0838104240 0x7f0838199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f08240b3bd0 0x7f08240b6090 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0838103880 0x7f08381990b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 --2- 172.21.15.193:0/1951715202 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0838102720 0x7f0838198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 >> 172.21.15.193:0/1951715202 conn(0x7f08380fdff0 msgr2=0x7f0838106c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 shutdown_connections 2024-08-20T06:34:17.322 INFO:teuthology.orchestra.run.smithi193.stderr:2024-08-20T06:34:17.319+0000 7f083db1e640 1 -- 172.21.15.193:0/1951715202 wait complete. 2024-08-20T06:34:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:16 smithi107 ceph-mon[18495]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:17.901 DEBUG:teuthology.orchestra.run.smithi107:mds.a> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.a.service 2024-08-20T06:34:17.903 DEBUG:teuthology.orchestra.run.smithi107:mds.d> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.d.service 2024-08-20T06:34:17.947 DEBUG:teuthology.orchestra.run.smithi107:mds.g> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.g.service 2024-08-20T06:34:17.989 DEBUG:teuthology.orchestra.run.smithi107:mds.j> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.j.service 2024-08-20T06:34:18.032 DEBUG:teuthology.orchestra.run.smithi183:mds.b> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.b.service 2024-08-20T06:34:18.035 DEBUG:teuthology.orchestra.run.smithi183:mds.e> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.e.service 2024-08-20T06:34:18.078 DEBUG:teuthology.orchestra.run.smithi183:mds.h> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.h.service 2024-08-20T06:34:18.121 DEBUG:teuthology.orchestra.run.smithi183:mds.k> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.k.service 2024-08-20T06:34:18.164 DEBUG:teuthology.orchestra.run.smithi193:mds.c> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.c.service 2024-08-20T06:34:18.167 DEBUG:teuthology.orchestra.run.smithi193:mds.f> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.f.service 2024-08-20T06:34:18.210 DEBUG:teuthology.orchestra.run.smithi193:mds.i> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.i.service 2024-08-20T06:34:18.253 DEBUG:teuthology.orchestra.run.smithi193:mds.l> sudo journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.l.service 2024-08-20T06:34:18.297 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-20T06:34:18.297 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:18.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:18.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:34:18.439 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.440 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:18 smithi193 ceph-mon[21944]: Deploying daemon mds.l on smithi193 2024-08-20T06:34:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:18.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:18.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:18 smithi107 ceph-mon[18495]: Deploying daemon mds.l on smithi193 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='client.24496 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: Saving service mds.all spec with placement smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:18.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:18 smithi183 ceph-mon[22204]: Deploying daemon mds.l on smithi193 2024-08-20T06:34:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.962+0000 7fadb77a8640 1 -- 172.21.15.107:0/3003810710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 msgr2=0x7fadb010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.962+0000 7fadb77a8640 1 --2- 172.21.15.107:0/3003810710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb010f290 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fada8009ca0 tx=0x7fada802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 -- 172.21.15.107:0/3003810710 shutdown_connections 2024-08-20T06:34:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 --2- 172.21.15.107:0/3003810710 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb010f290 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 --2- 172.21.15.107:0/3003810710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0104270 0x7fadb010c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 --2- 172.21.15.107:0/3003810710 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0103870 0x7fadb0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 -- 172.21.15.107:0/3003810710 >> 172.21.15.107:0/3003810710 conn(0x7fadb00fd440 msgr2=0x7fadb00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 -- 172.21.15.107:0/3003810710 shutdown_connections 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 -- 172.21.15.107:0/3003810710 wait complete. 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 Processor -- start 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.963+0000 7fadb77a8640 1 -- start start 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0103870 0x7fadb01016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 0x7fadb0101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:19.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01129a0 con 0x7fadb010ce20 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0112820 con 0x7fadb0104270 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb77a8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0112b20 con 0x7fadb0103870 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb5d45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 0x7fadb0101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb5d45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43120/0 (socket says 172.21.15.107:43120) 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb5d45640 1 -- 172.21.15.107:0/590939505 learned_addr learned my addr 172.21.15.107:0/590939505 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb5544640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0103870 0x7fadb01016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 -- 172.21.15.107:0/590939505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0103870 msgr2=0x7fadb01016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0103870 0x7fadb01016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 -- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 msgr2=0x7fadb0102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 -- 172.21.15.107:0/590939505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb01a7290 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb5d45640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.964+0000 7fadb4d43640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 0x7fadb0101c30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fada000d720 tx=0x7fada000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.965+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada0017070 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.965+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb01a7580 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.965+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fada000de30 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.965+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada00047a0 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.965+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a7ac0 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.966+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb0103c50 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.967+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fada00099d0 con 0x7fadb0104270 2024-08-20T06:34:19.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.968+0000 7fada67fc640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 0x7fad9c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.968+0000 7fadb5544640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 0x7fad9c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.968+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7fada00e37e0 con 0x7fadb0104270 2024-08-20T06:34:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.968+0000 7fadb5544640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 0x7fad9c0b6090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadac009bd0 tx=0x7fadac005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:19.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:18.971+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fada0020020 con 0x7fadb0104270 2024-08-20T06:34:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.182+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadb002e340 con 0x7fadb0104270 2024-08-20T06:34:19.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.183+0000 7fada67fc640 1 -- 172.21.15.107:0/590939505 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fada00ab230 con 0x7fadb0104270 2024-08-20T06:34:19.184 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:19.184 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":1,"btime":"2024-08-20T06:27:17:087657+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-20T06:34:19.184 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 1 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 msgr2=0x7fad9c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 0x7fad9c0b6090 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fadac009bd0 tx=0x7fadac005f90 comp rx=0 tx=0).stop 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 msgr2=0x7fadb0101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 0x7fadb0101c30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fada000d720 tx=0x7fada000dbe0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 shutdown_connections 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fadb010ce20 0x7fadb0102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fadb0104270 0x7fadb0101c30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fad9c0b3bd0 0x7fad9c0b6090 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 --2- 172.21.15.107:0/590939505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fadb0103870 0x7fadb01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 >> 172.21.15.107:0/590939505 conn(0x7fadb00fd440 msgr2=0x7fadb00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 shutdown_connections 2024-08-20T06:34:19.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:19.185+0000 7fadb77a8640 1 -- 172.21.15.107:0/590939505 wait complete. 2024-08-20T06:34:19.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:19 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/590939505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:19.613 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:19 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/590939505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:19.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:19 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/590939505' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:19.995 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:19 smithi193 systemd[1]: Starting Ceph mds.l for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:20.258 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:19 smithi193 podman[51941]: 2024-08-20 06:34:19.897281967 +0000 UTC m=+0.015986271 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:20.259 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 podman[51941]: 2024-08-20 06:34:20.022411741 +0000 UTC m=+0.141116046 container create f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-08-20T06:34:20.259 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 podman[51941]: 2024-08-20 06:34:20.258696285 +0000 UTC m=+0.377400589 container init f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-20T06:34:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-mon[21944]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6832/154109335,v1:172.21.15.193:6833/154109335] up:boot 2024-08-20T06:34:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-mon[21944]: fsmap 1 up:standby 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 podman[51941]: 2024-08-20 06:34:20.261493858 +0000 UTC m=+0.380198162 container start f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-mds[51959]: -- 172.21.15.193:0/3561611351 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ca4d221860 con 0x55ca4d1ff000 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l[51955]: starting mds.l at 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 bash[51941]: f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 2024-08-20T06:34:20.555 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:34:20 smithi193 systemd[1]: Started Ceph mds.l for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:20.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:20 smithi107 ceph-mon[18495]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:20.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:20 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6832/154109335,v1:172.21.15.193:6833/154109335] up:boot 2024-08-20T06:34:20.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:20 smithi107 ceph-mon[18495]: fsmap 1 up:standby 2024-08-20T06:34:20.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:20 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T06:34:20.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:20 smithi183 ceph-mon[22204]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:20.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:20 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6832/154109335,v1:172.21.15.193:6833/154109335] up:boot 2024-08-20T06:34:20.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:20 smithi183 ceph-mon[22204]: fsmap 1 up:standby 2024-08-20T06:34:20.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:20 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:21.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:21 smithi183 ceph-mon[22204]: Deploying daemon mds.k on smithi183 2024-08-20T06:34:21.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:21.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:21 smithi193 ceph-mon[21944]: Deploying daemon mds.k on smithi183 2024-08-20T06:34:21.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:21.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:21.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:21.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:21.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:21 smithi107 ceph-mon[18495]: Deploying daemon mds.k on smithi183 2024-08-20T06:34:21.937 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:22.193 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:22.442 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 systemd[1]: Starting Ceph mds.k for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:22.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:22 smithi107 ceph-mon[18495]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.625+0000 7fa0657bd640 1 -- 172.21.15.107:0/873199159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060103830 msgr2=0x7fa060103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.625+0000 7fa0657bd640 1 --2- 172.21.15.107:0/873199159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060103830 0x7fa060103c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa050009900 tx=0x7fa05002a410 comp rx=0 tx=0).stop 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- 172.21.15.107:0/873199159 shutdown_connections 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- 172.21.15.107:0/873199159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa06010cde0 0x7fa06010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- 172.21.15.107:0/873199159 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa06010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- 172.21.15.107:0/873199159 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa060103830 0x7fa060103c10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- 172.21.15.107:0/873199159 >> 172.21.15.107:0/873199159 conn(0x7fa0600fd400 msgr2=0x7fa0600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- 172.21.15.107:0/873199159 shutdown_connections 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- 172.21.15.107:0/873199159 wait complete. 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 Processor -- start 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- start start 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060103830 0x7fa060198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112950 con 0x7fa06010cde0 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa0601127d0 con 0x7fa060104230 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.626+0000 7fa0657bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa060112ad0 con 0x7fa060103830 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa057fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43138/0 (socket says 172.21.15.107:43138) 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa057fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:57102/0 (socket says 172.21.15.107:57102) 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 -- 172.21.15.107:0/1213260625 learned_addr learned my addr 172.21.15.107:0/1213260625 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05effd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060103830 0x7fa060198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 -- 172.21.15.107:0/1213260625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060103830 msgr2=0x7fa060198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060103830 0x7fa060198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 -- 172.21.15.107:0/1213260625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 msgr2=0x7fa060199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 -- 172.21.15.107:0/1213260625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06019efb0 con 0x7fa06010cde0 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa057fff640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05f7fe640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b970 tx=0x7fa05800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa058019070 con 0x7fa06010cde0 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06019f2a0 con 0x7fa06010cde0 2024-08-20T06:34:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa058014440 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0580075d0 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.627+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0601a59e0 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.628+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa060103c10 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.630+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fa05801c070 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.630+0000 7fa05cff9640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 0x7fa04c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.630+0000 7fa05effd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 0x7fa04c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.630+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7fa0580e2da0 con 0x7fa06010cde0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.631+0000 7fa05effd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 0x7fa04c0b6090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa050000c00 tx=0x7fa050005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:22.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.634+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa0580aa1a0 con 0x7fa06010cde0 2024-08-20T06:34:22.714 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:22 smithi183 ceph-mon[22204]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:22.715 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 podman[51017]: 2024-08-20 06:34:22.487572281 +0000 UTC m=+0.111023158 container create 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:34:22.715 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 podman[51017]: 2024-08-20 06:34:22.392302712 +0000 UTC m=+0.015753599 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:22.715 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 podman[51017]: 2024-08-20 06:34:22.715128625 +0000 UTC m=+0.338579501 container init 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-20T06:34:22.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:22 smithi193 ceph-mon[21944]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:22.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.852+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa06002e340 con 0x7fa06010cde0 2024-08-20T06:34:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.853+0000 7fa05cff9640 1 -- 172.21.15.107:0/1213260625 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fa0580ae6f0 con 0x7fa06010cde0 2024-08-20T06:34:22.854 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:22.854 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":2,"btime":"2024-08-20T06:34:20:289744+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-08-20T06:34:22.854 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 2 2024-08-20T06:34:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 msgr2=0x7fa04c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 0x7fa04c0b6090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa050000c00 tx=0x7fa050005a20 comp rx=0 tx=0).stop 2024-08-20T06:34:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 msgr2=0x7fa06019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa05800b970 tx=0x7fa05800be30 comp rx=0 tx=0).stop 2024-08-20T06:34:22.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 shutdown_connections 2024-08-20T06:34:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa06010cde0 0x7fa06019e7e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa060104230 0x7fa060199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa04c0b3bd0 0x7fa04c0b6090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 --2- 172.21.15.107:0/1213260625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa060103830 0x7fa060198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:22.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 >> 172.21.15.107:0/1213260625 conn(0x7fa0600fd400 msgr2=0x7fa0600fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:22.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 shutdown_connections 2024-08-20T06:34:22.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:22.856+0000 7fa0657bd640 1 -- 172.21.15.107:0/1213260625 wait complete. 2024-08-20T06:34:22.986 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 podman[51017]: 2024-08-20 06:34:22.718081738 +0000 UTC m=+0.341532615 container start 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True) 2024-08-20T06:34:22.986 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 ceph-mds[51036]: -- 172.21.15.183:0/549456812 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560824141860 con 0x56082411f000 2024-08-20T06:34:22.986 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k[51032]: starting mds.k at 2024-08-20T06:34:22.986 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 bash[51017]: 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 2024-08-20T06:34:22.986 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:34:22 smithi183 systemd[1]: Started Ceph mds.k for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1213260625' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:23 smithi183 ceph-mon[22204]: Deploying daemon mds.j on smithi107 2024-08-20T06:34:23.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1213260625' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:23.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:23.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:23.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:23.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:23 smithi193 ceph-mon[21944]: Deploying daemon mds.j on smithi107 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1213260625' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:23.829 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:23 smithi107 ceph-mon[18495]: Deploying daemon mds.j on smithi107 2024-08-20T06:34:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:24 smithi183 ceph-mon[22204]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:24 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6832/4048095343,v1:172.21.15.183:6833/4048095343] up:boot 2024-08-20T06:34:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:24 smithi183 ceph-mon[22204]: fsmap 2 up:standby 2024-08-20T06:34:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:24 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T06:34:24.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:24 smithi193 ceph-mon[21944]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:24.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:24 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6832/4048095343,v1:172.21.15.183:6833/4048095343] up:boot 2024-08-20T06:34:24.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:24 smithi193 ceph-mon[21944]: fsmap 2 up:standby 2024-08-20T06:34:24.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:24 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T06:34:24.852 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:24 smithi107 ceph-mon[18495]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:24.852 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:24 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6832/4048095343,v1:172.21.15.183:6833/4048095343] up:boot 2024-08-20T06:34:24.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:24 smithi107 ceph-mon[18495]: fsmap 2 up:standby 2024-08-20T06:34:24.853 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:24 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T06:34:25.541 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 systemd[1]: Starting Ceph mds.j for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:25.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:25.803 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 podman[58103]: 2024-08-20 06:34:25.442895227 +0000 UTC m=+0.015739721 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:25.803 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 podman[58103]: 2024-08-20 06:34:25.556350905 +0000 UTC m=+0.129195389 container create bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:34:25.803 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 podman[58103]: 2024-08-20 06:34:25.775500036 +0000 UTC m=+0.348344521 container init bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:34:25.803 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 podman[58103]: 2024-08-20 06:34:25.778578701 +0000 UTC m=+0.351423195 container start bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:34:25.939 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:26.062 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 ceph-mds[58148]: -- 172.21.15.107:0/1662468843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ce95cc5860 con 0x55ce95ca3400 2024-08-20T06:34:26.063 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j[58137]: starting mds.j at 2024-08-20T06:34:26.063 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 bash[58103]: bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f 2024-08-20T06:34:26.063 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:34:25 smithi107 systemd[1]: Started Ceph mds.j for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- 172.21.15.107:0/155720183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001019d0 msgr2=0x7f8500101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- 172.21.15.107:0/155720183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001019d0 0x7f8500101db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f84f0009900 tx=0x7f84f002a410 comp rx=0 tx=0).stop 2024-08-20T06:34:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- 172.21.15.107:0/155720183 shutdown_connections 2024-08-20T06:34:26.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- 172.21.15.107:0/155720183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8500107d60 0x7f850010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- 172.21.15.107:0/155720183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f85001073e0 0x7f8500107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- 172.21.15.107:0/155720183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001019d0 0x7f8500101db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- 172.21.15.107:0/155720183 >> 172.21.15.107:0/155720183 conn(0x7f85000fd420 msgr2=0x7f85000ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- 172.21.15.107:0/155720183 shutdown_connections 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- 172.21.15.107:0/155720183 wait complete. 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 Processor -- start 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 -- start start 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85001019d0 0x7f8500199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.409+0000 7f85069f9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8500107d60 0x7f85001a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f85069f9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f85001118d0 con 0x7f85001073e0 2024-08-20T06:34:26.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f85069f9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111750 con 0x7f8500107d60 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f85069f9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8500111a50 con 0x7f85001019d0 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40880/0 (socket says 172.21.15.107:40880) 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 -- 172.21.15.107:0/3203898114 learned_addr learned my addr 172.21.15.107:0/3203898114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f8504f96640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8500107d60 0x7f85001a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84fffff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85001019d0 0x7f8500199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 -- 172.21.15.107:0/3203898114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85001019d0 msgr2=0x7f8500199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85001019d0 0x7f8500199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 -- 172.21.15.107:0/3203898114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8500107d60 msgr2=0x7f85001a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8500107d60 0x7f85001a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 -- 172.21.15.107:0/3203898114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a5280 con 0x7f85001073e0 2024-08-20T06:34:26.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84f7fff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84e800d950 tx=0x7f84e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8017070 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a5450 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84e8009c30 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e800f620 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.410+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a59b0 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.411+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500101db0 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.413+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f84e8010460 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.413+0000 7f84fdffb640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 0x7f84ec0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.413+0000 7f84fffff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 0x7f84ec0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.414+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f84e80e3330 con 0x7f85001073e0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.414+0000 7f84fffff640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 0x7f84ec0b6090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:26.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.416+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f84e80aae20 con 0x7f85001073e0 2024-08-20T06:34:26.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.643+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f850006b640 con 0x7f85001073e0 2024-08-20T06:34:26.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.644+0000 7f84fdffb640 1 -- 172.21.15.107:0/3203898114 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3091 (secure 0 0 0) 0x7f84e8009de0 con 0x7f85001073e0 2024-08-20T06:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:26.645 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":4,"btime":"2024-08-20T06:34:26:449815+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-20T06:34:26.645 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 4 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 msgr2=0x7f84ec0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 0x7f84ec0b6090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84f0006010 tx=0x7f84f0000bc0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 msgr2=0x7f850019a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84e800d950 tx=0x7f84e800de10 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 shutdown_connections 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8500107d60 0x7f85001a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f85001073e0 0x7f850019a200 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84ec0b3bd0 0x7f84ec0b6090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 --2- 172.21.15.107:0/3203898114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85001019d0 0x7f8500199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 >> 172.21.15.107:0/3203898114 conn(0x7f85000fd420 msgr2=0x7f85000ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 shutdown_connections 2024-08-20T06:34:26.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:26.646+0000 7f85069f9640 1 -- 172.21.15.107:0/3203898114 wait complete. 2024-08-20T06:34:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: Deploying daemon mds.d on smithi107 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.107:6834/3097875231,v1:172.21.15.107:6835/3097875231] up:boot 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: fsmap 3 up:standby 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T06:34:27.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:26 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3203898114' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:27.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: Deploying daemon mds.d on smithi107 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.107:6834/3097875231,v1:172.21.15.107:6835/3097875231] up:boot 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: fsmap 3 up:standby 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T06:34:27.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:26 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3203898114' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: Deploying daemon mds.d on smithi107 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.107:6834/3097875231,v1:172.21.15.107:6835/3097875231] up:boot 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: fsmap 3 up:standby 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T06:34:27.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:26 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3203898114' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:28.861 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:28 smithi107 systemd[1]: Starting Ceph mds.d for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:29.195 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:28 smithi107 ceph-mon[18495]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:29.195 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:28 smithi107 podman[58519]: 2024-08-20 06:34:28.846293972 +0000 UTC m=+0.015372862 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:29.195 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:28 smithi107 podman[58519]: 2024-08-20 06:34:28.975058419 +0000 UTC m=+0.144137312 container create 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-20T06:34:29.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:28 smithi183 ceph-mon[22204]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:29.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:28 smithi193 ceph-mon[21944]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:29.418 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:29.611 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 podman[58519]: 2024-08-20 06:34:29.194611646 +0000 UTC m=+0.363690536 container init 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T06:34:29.611 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 podman[58519]: 2024-08-20 06:34:29.197554807 +0000 UTC m=+0.366633707 container start 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:34:29.611 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 ceph-mds[58538]: -- 172.21.15.107:0/1634207706 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5649d196f860 con 0x5649d194d400 2024-08-20T06:34:29.611 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d[58534]: starting mds.d at 2024-08-20T06:34:29.611 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 bash[58519]: 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 2024-08-20T06:34:29.612 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:34:29 smithi107 systemd[1]: Started Ceph mds.d for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:29.664 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:30.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 -- 172.21.15.107:0/1544955437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f141073a0 msgr2=0x7f0f141077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:30.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/1544955437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f141073a0 0x7f0f141077e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0efc009900 tx=0x7f0efc02a410 comp rx=0 tx=0).stop 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 -- 172.21.15.107:0/1544955437 shutdown_connections 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/1544955437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 0x7f0f1410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/1544955437 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f141073a0 0x7f0f141077e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/1544955437 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f14101990 0x7f0f14101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 -- 172.21.15.107:0/1544955437 >> 172.21.15.107:0/1544955437 conn(0x7f0f140fd400 msgr2=0x7f0f140ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 -- 172.21.15.107:0/1544955437 shutdown_connections 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 -- 172.21.15.107:0/1544955437 wait complete. 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.145+0000 7f0f1a10a640 1 Processor -- start 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 -- start start 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f141073a0 0x7f0f1419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:30.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 0x7f0f1419a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f14111850 con 0x7f0f14101990 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141116d0 con 0x7f0f141073a0 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f1a10a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f141119d0 con 0x7f0f14107d20 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40908/0 (socket says 172.21.15.107:40908) 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 -- 172.21.15.107:0/2329285157 learned_addr learned my addr 172.21.15.107:0/2329285157 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f12ffd640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f141073a0 0x7f0f1419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f13fff640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 0x7f0f1419a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 -- 172.21.15.107:0/2329285157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 msgr2=0x7f0f1419a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 0x7f0f1419a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 -- 172.21.15.107:0/2329285157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f141073a0 msgr2=0x7f0f1419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:30.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f141073a0 0x7f0f1419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 -- 172.21.15.107:0/2329285157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f141a50c0 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.146+0000 7f0f137fe640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400d950 tx=0x7f0f0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.147+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f04017030 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.147+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f141a53b0 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.147+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f04004020 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.147+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f040055d0 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.147+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f141a58c0 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.148+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f14101d70 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.149+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f0f04009a30 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.150+0000 7f0f10ff9640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 0x7f0f000b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.150+0000 7f0f12ffd640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 0x7f0f000b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.150+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f0f040e37b0 con 0x7f0f14101990 2024-08-20T06:34:30.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.150+0000 7f0f12ffd640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 0x7f0f000b6090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0057c0 tx=0x7f0efc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:30.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.153+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0f040ab2a0 con 0x7f0f14101990 2024-08-20T06:34:30.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.365+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f1406abe0 con 0x7f0f14101990 2024-08-20T06:34:30.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.366+0000 7f0f10ff9640 1 -- 172.21.15.107:0/2329285157 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3919 (secure 0 0 0) 0x7f0f1406abe0 con 0x7f0f14101990 2024-08-20T06:34:30.367 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:30.368 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":5,"btime":"2024-08-20T06:34:29:923913+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-20T06:34:30.368 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 5 2024-08-20T06:34:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 msgr2=0x7f0f000b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 0x7f0f000b6090 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f0efc0057c0 tx=0x7f0efc0058d0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 msgr2=0x7f0f14199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f0400d950 tx=0x7f0f0400de10 comp rx=0 tx=0).stop 2024-08-20T06:34:30.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 shutdown_connections 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f000b3bd0 0x7f0f000b6090 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f14107d20 0x7f0f1419a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f141073a0 0x7f0f1419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 --2- 172.21.15.107:0/2329285157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f14101990 0x7f0f14199cc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 >> 172.21.15.107:0/2329285157 conn(0x7f0f140fd400 msgr2=0x7f0f140ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 shutdown_connections 2024-08-20T06:34:30.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:30.368+0000 7f0f1a10a640 1 -- 172.21.15.107:0/2329285157 wait complete. 2024-08-20T06:34:30.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:30.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.437 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: Deploying daemon mds.e on smithi183 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.107:6836/3157081894,v1:172.21.15.107:6837/3157081894] up:boot 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: fsmap 4 up:standby 2024-08-20T06:34:30.438 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:30 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T06:34:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: Deploying daemon mds.e on smithi183 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.107:6836/3157081894,v1:172.21.15.107:6837/3157081894] up:boot 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: fsmap 4 up:standby 2024-08-20T06:34:30.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:30 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T06:34:30.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: Deploying daemon mds.e on smithi183 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.107:6836/3157081894,v1:172.21.15.107:6837/3157081894] up:boot 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: fsmap 4 up:standby 2024-08-20T06:34:30.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:30 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T06:34:31.236 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 systemd[1]: Starting Ceph mds.e for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:31.609 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:31 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2329285157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:31.610 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 podman[51317]: 2024-08-20 06:34:31.275131356 +0000 UTC m=+0.016408797 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:31.610 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 podman[51317]: 2024-08-20 06:34:31.382003041 +0000 UTC m=+0.123280481 container create 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:34:31.610 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 podman[51317]: 2024-08-20 06:34:31.609605251 +0000 UTC m=+0.350882691 container init 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T06:34:31.610 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:31 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2329285157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:31.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:31 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2329285157' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:31.986 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 podman[51317]: 2024-08-20 06:34:31.612865714 +0000 UTC m=+0.354143163 container start 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-08-20T06:34:31.986 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 ceph-mds[51336]: -- 172.21.15.183:0/3578989282 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a0457c9860 con 0x55a0457a6800 2024-08-20T06:34:31.986 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: starting mds.e at 2024-08-20T06:34:31.986 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 bash[51317]: 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc 2024-08-20T06:34:31.986 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:34:31 smithi183 systemd[1]: Started Ceph mds.e for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:32.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:32.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: Deploying daemon mds.f on smithi193 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6834/2773184516,v1:172.21.15.183:6835/2773184516] up:boot 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: fsmap 5 up:standby 2024-08-20T06:34:32.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:32 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: Deploying daemon mds.f on smithi193 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6834/2773184516,v1:172.21.15.183:6835/2773184516] up:boot 2024-08-20T06:34:33.041 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: fsmap 5 up:standby 2024-08-20T06:34:33.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:32 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:34:33.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:33.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:33.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:33.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: Deploying daemon mds.f on smithi193 2024-08-20T06:34:33.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6834/2773184516,v1:172.21.15.183:6835/2773184516] up:boot 2024-08-20T06:34:33.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: fsmap 5 up:standby 2024-08-20T06:34:33.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:32 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:34:33.315 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:33.743 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 systemd[1]: Starting Ceph mds.f for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 -- 172.21.15.107:0/3448043634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 msgr2=0x7f52e810f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 --2- 172.21.15.107:0/3448043634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e810f270 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009ca0 tx=0x7f52e002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 -- 172.21.15.107:0/3448043634 shutdown_connections 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 --2- 172.21.15.107:0/3448043634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e810f270 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 --2- 172.21.15.107:0/3448043634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81042d0 0x7f52e810c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 --2- 172.21.15.107:0/3448043634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81039b0 0x7f52e8103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 -- 172.21.15.107:0/3448043634 >> 172.21.15.107:0/3448043634 conn(0x7f52e80fd430 msgr2=0x7f52e80ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 -- 172.21.15.107:0/3448043634 shutdown_connections 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.808+0000 7f52ef362640 1 -- 172.21.15.107:0/3448043634 wait complete. 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 Processor -- start 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 -- start start 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81039b0 0x7f52e8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 0x7f52e81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f52e8112980 con 0x7f52e810ce10 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f52e8112800 con 0x7f52e81042d0 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ef362640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f52e8112b00 con 0x7f52e81039b0 2024-08-20T06:34:33.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ed8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ed8ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40932/0 (socket says 172.21.15.107:40932) 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ed8ff640 1 -- 172.21.15.107:0/2645115250 learned_addr learned my addr 172.21.15.107:0/2645115250 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ed0fe640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81039b0 0x7f52e8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.809+0000 7f52ec8fd640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 0x7f52e81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 -- 172.21.15.107:0/2645115250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81039b0 msgr2=0x7f52e8198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81039b0 0x7f52e8198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 -- 172.21.15.107:0/2645115250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 msgr2=0x7f52e81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 0x7f52e81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 -- 172.21.15.107:0/2645115250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e819ef40 con 0x7f52e810ce10 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ec8fd640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 0x7f52e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ed8ff640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005cc0 tx=0x7f52e0032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0032e00 con 0x7f52e810ce10 2024-08-20T06:34:33.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f52e00044d0 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e819f1d0 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0030b70 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.810+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e81a58f0 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.811+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8103d90 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.813+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f52e0030d10 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.814+0000 7f52de7fc640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 0x7f52d40b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.814+0000 7f52ed0fe640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 0x7f52d40b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.814+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f52e0104da0 con 0x7f52e810ce10 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.814+0000 7f52ed0fe640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 0x7f52d40b6110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52e40058c0 tx=0x7f52e400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:33.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:33.817+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f52e00cc890 con 0x7f52e810ce10 2024-08-20T06:34:33.997 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 podman[52240]: 2024-08-20 06:34:33.742688495 +0000 UTC m=+0.111934911 container create efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:34:33.997 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 podman[52240]: 2024-08-20 06:34:33.646881315 +0000 UTC m=+0.016127732 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:33.997 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 podman[52240]: 2024-08-20 06:34:33.970061421 +0000 UTC m=+0.339307832 container init efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-20T06:34:33.997 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 podman[52240]: 2024-08-20 06:34:33.972910766 +0000 UTC m=+0.342157177 container start efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T06:34:33.997 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 ceph-mds[52259]: -- 172.21.15.193:0/3598393622 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55944034d860 con 0x55944032b000 2024-08-20T06:34:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.027+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52e8199fc0 con 0x7f52e810ce10 2024-08-20T06:34:34.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.028+0000 7f52de7fc640 1 -- 172.21.15.107:0/2645115250 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4747 (secure 0 0 0) 0x7f52e8199fc0 con 0x7f52e810ce10 2024-08-20T06:34:34.029 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:34.029 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":6,"btime":"2024-08-20T06:34:32:352350+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-20T06:34:34.029 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 6 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 msgr2=0x7f52d40b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 0x7f52d40b6110 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f52e40058c0 tx=0x7f52e400a2d0 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 msgr2=0x7f52e819e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f52e0005cc0 tx=0x7f52e0032a10 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 shutdown_connections 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f52e810ce10 0x7f52e819e770 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f52e81042d0 0x7f52e81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f52d40b3c50 0x7f52d40b6110 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 --2- 172.21.15.107:0/2645115250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52e81039b0 0x7f52e8198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 >> 172.21.15.107:0/2645115250 conn(0x7f52e80fd430 msgr2=0x7f52e80fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 shutdown_connections 2024-08-20T06:34:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:34.030+0000 7f52ef362640 1 -- 172.21.15.107:0/2645115250 wait complete. 2024-08-20T06:34:34.304 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: starting mds.f at 2024-08-20T06:34:34.304 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:34 smithi193 bash[52240]: efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 2024-08-20T06:34:34.304 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:34:34 smithi193 systemd[1]: Started Ceph mds.f for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:34.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:34.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2645115250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:34.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:34.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:34 smithi193 ceph-mon[21944]: Deploying daemon mds.c on smithi193 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2645115250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:35.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:35.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:35.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:34 smithi107 ceph-mon[18495]: Deploying daemon mds.c on smithi193 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2645115250' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:35.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:34 smithi183 ceph-mon[22204]: Deploying daemon mds.c on smithi193 2024-08-20T06:34:36.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:35 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6834/2974563037,v1:172.21.15.193:6835/2974563037] up:boot 2024-08-20T06:34:36.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:35 smithi193 ceph-mon[21944]: fsmap 6 up:standby 2024-08-20T06:34:36.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:35 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:34:36.055 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:35 smithi193 systemd[1]: Starting Ceph mds.c for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:35 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6834/2974563037,v1:172.21.15.193:6835/2974563037] up:boot 2024-08-20T06:34:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:35 smithi107 ceph-mon[18495]: fsmap 6 up:standby 2024-08-20T06:34:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:35 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:34:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:35 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6834/2974563037,v1:172.21.15.193:6835/2974563037] up:boot 2024-08-20T06:34:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:35 smithi183 ceph-mon[22204]: fsmap 6 up:standby 2024-08-20T06:34:36.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:35 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:34:36.351 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 podman[52542]: 2024-08-20 06:34:36.088319277 +0000 UTC m=+0.107794919 container create 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 podman[52542]: 2024-08-20 06:34:35.996131738 +0000 UTC m=+0.015607334 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 podman[52542]: 2024-08-20 06:34:36.292751931 +0000 UTC m=+0.312227527 container init 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 podman[52542]: 2024-08-20 06:34:36.295594071 +0000 UTC m=+0.315069668 container start 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0) 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mds[52561]: -- 172.21.15.193:0/1158540405 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56283e60b860 con 0x56283e5e9400 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: starting mds.c at 2024-08-20T06:34:36.352 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 bash[52542]: 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a 2024-08-20T06:34:36.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:36.778 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:34:36 smithi193 systemd[1]: Started Ceph mds.c for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:36.986 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:37.054 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:36 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:37.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:36 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:37.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:37.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:37.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:37.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:37.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:36 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 -- 172.21.15.107:0/1707642693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c10cdc0 msgr2=0x7feb3c10f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 --2- 172.21.15.107:0/1707642693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c10cdc0 0x7feb3c10f260 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb34009ca0 tx=0x7feb3402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 -- 172.21.15.107:0/1707642693 shutdown_connections 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 --2- 172.21.15.107:0/1707642693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c10cdc0 0x7feb3c10f260 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 --2- 172.21.15.107:0/1707642693 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101f00 0x7feb3c10c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 --2- 172.21.15.107:0/1707642693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c101500 0x7feb3c1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 -- 172.21.15.107:0/1707642693 >> 172.21.15.107:0/1707642693 conn(0x7feb3c076bd0 msgr2=0x7feb3c076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:37.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 -- 172.21.15.107:0/1707642693 shutdown_connections 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.399+0000 7feb411c5640 1 -- 172.21.15.107:0/1707642693 wait complete. 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.400+0000 7feb411c5640 1 Processor -- start 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.400+0000 7feb411c5640 1 -- start start 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 0x7feb3c19b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c10cdc0 0x7feb3c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1128d0 con 0x7feb3c101f00 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112750 con 0x7feb3c101500 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb411c5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112a50 con 0x7feb3c10cdc0 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33594/0 (socket says 172.21.15.107:33594) 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 -- 172.21.15.107:0/3660659207 learned_addr learned my addr 172.21.15.107:0/3660659207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:37.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3ad9d640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 0x7feb3c19b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3b59e640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c10cdc0 0x7feb3c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 -- 172.21.15.107:0/3660659207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c10cdc0 msgr2=0x7feb3c1a09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c10cdc0 0x7feb3c1a09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 -- 172.21.15.107:0/3660659207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 msgr2=0x7feb3c19b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 0x7feb3c19b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3a59c640 1 -- 172.21.15.107:0/3660659207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c1a1170 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.401+0000 7feb3ad9d640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 0x7feb3c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb3a59c640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb2c005fd0 tx=0x7feb2c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c017070 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c1a1460 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a7cf0 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feb2c009c30 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.402+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c003c70 con 0x7feb3c101f00 2024-08-20T06:34:37.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.403+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb280052c0 con 0x7feb3c101f00 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.404+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7feb2c003e10 con 0x7feb3c101f00 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.405+0000 7feb23fff640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 0x7feb240b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.405+0000 7feb3ad9d640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 0x7feb240b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.405+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7feb2c0e31a0 con 0x7feb3c101f00 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.405+0000 7feb3ad9d640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 0x7feb240b6090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb300058c0 tx=0x7feb3000a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:37.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.409+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7feb2c020030 con 0x7feb3c101f00 2024-08-20T06:34:37.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.638+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb28005880 con 0x7feb3c101f00 2024-08-20T06:34:37.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.638+0000 7feb23fff640 1 -- 172.21.15.107:0/3660659207 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7feb2c0aac90 con 0x7feb3c101f00 2024-08-20T06:34:37.639 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:37.639 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":8,"btime":"2024-08-20T06:34:36:777252+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-20T06:34:37.640 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 8 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 msgr2=0x7feb240b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 0x7feb240b6090 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7feb300058c0 tx=0x7feb3000a250 comp rx=0 tx=0).stop 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 msgr2=0x7feb3c19b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7feb2c005fd0 tx=0x7feb2c00dde0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 shutdown_connections 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feb240b3bd0 0x7feb240b6090 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb3c10cdc0 0x7feb3c1a09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feb3c101f00 0x7feb3c19b600 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 --2- 172.21.15.107:0/3660659207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feb3c101500 0x7feb3c19b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:37.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 >> 172.21.15.107:0/3660659207 conn(0x7feb3c076bd0 msgr2=0x7feb3c100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:37.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 shutdown_connections 2024-08-20T06:34:37.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:37.641+0000 7feb411c5640 1 -- 172.21.15.107:0/3660659207 wait complete. 2024-08-20T06:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:37 smithi183 ceph-mon[22204]: Deploying daemon mds.h on smithi183 2024-08-20T06:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:37 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6836/2519256232,v1:172.21.15.193:6837/2519256232] up:boot 2024-08-20T06:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:37 smithi183 ceph-mon[22204]: fsmap 7 up:standby 2024-08-20T06:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:37 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:34:37.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:37 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3660659207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:37 smithi193 ceph-mon[21944]: Deploying daemon mds.h on smithi183 2024-08-20T06:34:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:37 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6836/2519256232,v1:172.21.15.193:6837/2519256232] up:boot 2024-08-20T06:34:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:37 smithi193 ceph-mon[21944]: fsmap 7 up:standby 2024-08-20T06:34:38.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:37 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:34:38.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:37 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3660659207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:37 smithi107 ceph-mon[18495]: Deploying daemon mds.h on smithi183 2024-08-20T06:34:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:37 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6836/2519256232,v1:172.21.15.193:6837/2519256232] up:boot 2024-08-20T06:34:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:37 smithi107 ceph-mon[18495]: fsmap 7 up:standby 2024-08-20T06:34:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:37 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:34:38.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:37 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3660659207' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:38.399 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 systemd[1]: Starting Ceph mds.h for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:38.663 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 podman[51621]: 2024-08-20 06:34:38.301686676 +0000 UTC m=+0.015714851 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:38.663 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 podman[51621]: 2024-08-20 06:34:38.409791981 +0000 UTC m=+0.123820153 container create 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:34:38.663 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 podman[51621]: 2024-08-20 06:34:38.637267066 +0000 UTC m=+0.351295249 container init 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-08-20T06:34:38.663 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 podman[51621]: 2024-08-20 06:34:38.640065617 +0000 UTC m=+0.354093791 container start 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-08-20T06:34:38.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:38 smithi183 ceph-mon[22204]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:38.986 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 ceph-mds[51641]: -- 172.21.15.183:0/1401651342 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c6e173860 con 0x556c6e151000 2024-08-20T06:34:38.986 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: starting mds.h at 2024-08-20T06:34:38.986 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 bash[51621]: 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf 2024-08-20T06:34:38.987 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:34:38 smithi183 systemd[1]: Started Ceph mds.h for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:39.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:38 smithi193 ceph-mon[21944]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:39.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:38 smithi107 ceph-mon[18495]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:40.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:boot 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: fsmap 8 up:standby 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:40.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:39 smithi193 ceph-mon[21944]: Deploying daemon mds.i on smithi193 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:boot 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: fsmap 8 up:standby 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:40.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:40.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:40.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:39 smithi107 ceph-mon[18495]: Deploying daemon mds.i on smithi193 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:boot 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: fsmap 8 up:standby 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:40.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:39 smithi183 ceph-mon[22204]: Deploying daemon mds.i on smithi193 2024-08-20T06:34:40.378 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:40.639 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:40.804 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:40 smithi193 systemd[1]: Starting Ceph mds.i for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:41.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:40 smithi107 ceph-mon[18495]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:41.110 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 -- 172.21.15.107:0/1376305160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 msgr2=0x7f14fc10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 --2- 172.21.15.107:0/1376305160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc10f640 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009ca0 tx=0x7f14f402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 -- 172.21.15.107:0/1376305160 shutdown_connections 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 --2- 172.21.15.107:0/1376305160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc10f640 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 --2- 172.21.15.107:0/1376305160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 0x7f14fc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 --2- 172.21.15.107:0/1376305160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 -- 172.21.15.107:0/1376305160 >> 172.21.15.107:0/1376305160 conn(0x7f14fc0fe500 msgr2=0x7f14fc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 -- 172.21.15.107:0/1376305160 shutdown_connections 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.098+0000 7f1503ca9640 1 -- 172.21.15.107:0/1376305160 wait complete. 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 Processor -- start 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 -- start start 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 0x7f14fc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:41.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc1129c0 con 0x7f14fc108e40 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112840 con 0x7f14fc1084c0 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1503ca9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f14fc112b40 con 0x7f14fc102ab0 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1502246640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1502246640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33624/0 (socket says 172.21.15.107:33624) 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1502246640 1 -- 172.21.15.107:0/3326083274 learned_addr learned my addr 172.21.15.107:0/3326083274 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.099+0000 7f1501a45640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 -- 172.21.15.107:0/3326083274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 msgr2=0x7f14fc198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1501244640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 0x7f14fc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 -- 172.21.15.107:0/3326083274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 msgr2=0x7f14fc199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 0x7f14fc199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 -- 172.21.15.107:0/3326083274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14fc19f000 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1501a45640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1502246640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005cc0 tx=0x7f14f4032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4004040 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14fc19f290 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f14f4004670 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4030cc0 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.100+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14fc1a59b0 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.101+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14fc102e90 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.102+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f14f40041e0 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.103+0000 7f14f2ffd640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 0x7f14e80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.103+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f14f4105060 con 0x7f14fc108e40 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.103+0000 7f1501a45640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 0x7f14e80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.103+0000 7f1501a45640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 0x7f14e80b6090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14f80058c0 tx=0x7f14f8005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:41.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.106+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f14f40ccb50 con 0x7f14fc108e40 2024-08-20T06:34:41.145 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:40 smithi193 ceph-mon[21944]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:41.145 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:40 smithi193 podman[52842]: 2024-08-20 06:34:40.811276592 +0000 UTC m=+0.016008349 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:41.145 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:40 smithi193 podman[52842]: 2024-08-20 06:34:40.909589612 +0000 UTC m=+0.114321371 container create 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-20T06:34:41.145 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 podman[52842]: 2024-08-20 06:34:41.145556013 +0000 UTC m=+0.350287768 container init 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:34:41.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:40 smithi183 ceph-mon[22204]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:41.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.319+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14fc10ac20 con 0x7f14fc108e40 2024-08-20T06:34:41.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.319+0000 7f14f2ffd640 1 -- 172.21.15.107:0/3326083274 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7f14f40cc450 con 0x7f14fc108e40 2024-08-20T06:34:41.320 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:41.320 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":9,"btime":"2024-08-20T06:34:38:788914+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-20T06:34:41.321 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 9 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 msgr2=0x7f14e80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 0x7f14e80b6090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f14f80058c0 tx=0x7f14f8005ec0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 msgr2=0x7f14fc19e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f14f4005cc0 tx=0x7f14f4032c40 comp rx=0 tx=0).stop 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 shutdown_connections 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f14fc108e40 0x7f14fc19e830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f14fc1084c0 0x7f14fc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f14e80b3bd0 0x7f14e80b6090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 --2- 172.21.15.107:0/3326083274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14fc102ab0 0x7f14fc198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 >> 172.21.15.107:0/3326083274 conn(0x7f14fc0fe500 msgr2=0x7f14fc0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 shutdown_connections 2024-08-20T06:34:41.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:41.322+0000 7f1503ca9640 1 -- 172.21.15.107:0/3326083274 wait complete. 2024-08-20T06:34:41.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 podman[52842]: 2024-08-20 06:34:41.148533928 +0000 UTC m=+0.353265691 container start 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:34:41.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 ceph-mds[52861]: -- 172.21.15.193:0/3103840534 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef8f325860 con 0x55ef8f303000 2024-08-20T06:34:41.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: starting mds.i at 2024-08-20T06:34:41.555 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 bash[52842]: 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa 2024-08-20T06:34:41.555 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:34:41 smithi193 systemd[1]: Started Ceph mds.i for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: Deploying daemon mds.g on smithi107 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3326083274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:boot 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: fsmap 9 up:standby 2024-08-20T06:34:42.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:42 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: Deploying daemon mds.g on smithi107 2024-08-20T06:34:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3326083274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:boot 2024-08-20T06:34:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: fsmap 9 up:standby 2024-08-20T06:34:42.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:42 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:34:42.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:42.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: Deploying daemon mds.g on smithi107 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3326083274' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:boot 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: fsmap 9 up:standby 2024-08-20T06:34:42.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:42 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:34:43.861 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:43 smithi107 systemd[1]: Starting Ceph mds.g for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:44.094 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:44.202 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:43 smithi107 podman[59290]: 2024-08-20 06:34:43.849942188 +0000 UTC m=+0.015531952 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:44.202 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:43 smithi107 podman[59290]: 2024-08-20 06:34:43.981859432 +0000 UTC m=+0.147449173 container create 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, io.buildah.version=1.36.0) 2024-08-20T06:34:44.202 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 podman[59290]: 2024-08-20 06:34:44.201711446 +0000 UTC m=+0.367301208 container init 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T06:34:44.404 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:44 smithi107 ceph-mon[18495]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 podman[59290]: 2024-08-20 06:34:44.204736908 +0000 UTC m=+0.370326651 container start 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 ceph-mds[59327]: -- 172.21.15.107:0/399108005 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a58000f860 con 0x55a57ffec800 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g[59305]: starting mds.g at 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 bash[59290]: 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 2024-08-20T06:34:44.526 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:34:44 smithi107 systemd[1]: Started Ceph mds.g for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:44 smithi183 ceph-mon[22204]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:44.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:44 smithi193 ceph-mon[21944]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:44.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 -- 172.21.15.107:0/922989456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8102ad0 msgr2=0x7f99a8102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:44.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 --2- 172.21.15.107:0/922989456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8102ad0 0x7f99a8102eb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9998009900 tx=0x7f999802a410 comp rx=0 tx=0).stop 2024-08-20T06:34:44.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 -- 172.21.15.107:0/922989456 shutdown_connections 2024-08-20T06:34:44.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 --2- 172.21.15.107:0/922989456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8108e60 0x7f99a810f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:44.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 --2- 172.21.15.107:0/922989456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 0x7f99a8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:44.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 --2- 172.21.15.107:0/922989456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8102ad0 0x7f99a8102eb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:44.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 -- 172.21.15.107:0/922989456 >> 172.21.15.107:0/922989456 conn(0x7f99a80fe540 msgr2=0x7f99a8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:44.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 -- 172.21.15.107:0/922989456 shutdown_connections 2024-08-20T06:34:44.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.867+0000 7f99ad24a640 1 -- 172.21.15.107:0/922989456 wait complete. 2024-08-20T06:34:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 Processor -- start 2024-08-20T06:34:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 -- start start 2024-08-20T06:34:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 0x7f99a80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 0x7f99a8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:44.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112a10 con 0x7f99a8108e60 2024-08-20T06:34:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112890 con 0x7f99a8102ad0 2024-08-20T06:34:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99ad24a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f99a8112b90 con 0x7f99a81084e0 2024-08-20T06:34:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99a759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:44.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99a759e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33670/0 (socket says 172.21.15.107:33670) 2024-08-20T06:34:44.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.868+0000 7f99a759e640 1 -- 172.21.15.107:0/3757809728 learned_addr learned my addr 172.21.15.107:0/3757809728 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:44.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a6d9d640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 0x7f99a80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:44.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f999ffff640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 0x7f99a8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:44.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 -- 172.21.15.107:0/3757809728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 msgr2=0x7f99a8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:44.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 0x7f99a8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 -- 172.21.15.107:0/3757809728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 msgr2=0x7f99a80751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 0x7f99a80751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 -- 172.21.15.107:0/3757809728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a81a7420 con 0x7f99a8108e60 2024-08-20T06:34:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a6d9d640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 0x7f99a80751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:34:44.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a759e640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b970 tx=0x7f99a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a0019070 con 0x7f99a8108e60 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f99a0014440 con 0x7f99a8108e60 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a81a7710 con 0x7f99a8108e60 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99a00075d0 con 0x7f99a8108e60 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.869+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a81a7c80 con 0x7f99a8108e60 2024-08-20T06:34:44.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.870+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99a8102f30 con 0x7f99a8108e60 2024-08-20T06:34:44.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.872+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f99a001c070 con 0x7f99a8108e60 2024-08-20T06:34:44.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.873+0000 7f99a4d99640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 0x7f99940b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:44.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.873+0000 7f99a6d9d640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 0x7f99940b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:44.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.873+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f99a00e3220 con 0x7f99a8108e60 2024-08-20T06:34:44.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.873+0000 7f99a6d9d640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 0x7f99940b6090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f9998005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:44.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:44.876+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f99a00aad10 con 0x7f99a8108e60 2024-08-20T06:34:45.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.097+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99a802e340 con 0x7f99a8108e60 2024-08-20T06:34:45.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.097+0000 7f99a4d99640 1 -- 172.21.15.107:0/3757809728 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8890 (secure 0 0 0) 0x7f99a00aa610 con 0x7f99a8108e60 2024-08-20T06:34:45.099 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:45.099 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 11 2024-08-20T06:34:45.099 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":11,"btime":"2024-08-20T06:34:44:305291+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 msgr2=0x7f99940b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 0x7f99940b6090 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9998000c00 tx=0x7f9998005a20 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 msgr2=0x7f99a8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f99a000b970 tx=0x7f99a000be30 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 shutdown_connections 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f99a8108e60 0x7f99a8075c50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f99940b3bd0 0x7f99940b6090 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99a81084e0 0x7f99a8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 --2- 172.21.15.107:0/3757809728 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f99a8102ad0 0x7f99a80751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.099+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 >> 172.21.15.107:0/3757809728 conn(0x7f99a80fe540 msgr2=0x7f99a8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.100+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 shutdown_connections 2024-08-20T06:34:45.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:45.100+0000 7f99ad24a640 1 -- 172.21.15.107:0/3757809728 wait complete. 2024-08-20T06:34:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.107:6838/1902182493,v1:172.21.15.107:6839/1902182493] up:boot 2024-08-20T06:34:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: fsmap 10 up:standby 2024-08-20T06:34:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: Deploying daemon mds.a on smithi107 2024-08-20T06:34:45.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:45 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3757809728' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:45.637 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.107:6838/1902182493,v1:172.21.15.107:6839/1902182493] up:boot 2024-08-20T06:34:45.637 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: fsmap 10 up:standby 2024-08-20T06:34:45.637 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: Deploying daemon mds.a on smithi107 2024-08-20T06:34:45.638 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:45 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3757809728' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.107:6838/1902182493,v1:172.21.15.107:6839/1902182493] up:boot 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: fsmap 10 up:standby 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:45.706 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:45.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:45.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: Deploying daemon mds.a on smithi107 2024-08-20T06:34:45.707 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:45 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3757809728' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:46.717 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:46 smithi193 ceph-mon[21944]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:46.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:46 smithi183 ceph-mon[22204]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:46.749 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:46 smithi107 ceph-mon[18495]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:47.361 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 systemd[1]: Starting Ceph mds.a for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:47.680 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 podman[59706]: 2024-08-20 06:34:47.32342558 +0000 UTC m=+0.015818600 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:47.680 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 podman[59706]: 2024-08-20 06:34:47.443582086 +0000 UTC m=+0.135975136 container create 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-20T06:34:47.680 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 podman[59706]: 2024-08-20 06:34:47.679570155 +0000 UTC m=+0.371963168 container init 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-20T06:34:47.823 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:48.074 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 podman[59706]: 2024-08-20 06:34:47.682490092 +0000 UTC m=+0.374883106 container start 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a, io.buildah.version=1.36.0, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:34:48.074 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 ceph-mds[59724]: -- 172.21.15.107:0/4165597137 <== mon.0 v2:172.21.15.107:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dee9fad860 con 0x55dee9f8b400 2024-08-20T06:34:48.074 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a[59720]: starting mds.a at 2024-08-20T06:34:48.074 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 bash[59706]: 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 2024-08-20T06:34:48.074 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:34:47 smithi107 systemd[1]: Started Ceph mds.a for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:48.076 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 -- 172.21.15.107:0/793645341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 msgr2=0x7fd13410f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 --2- 172.21.15.107:0/793645341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd13410f2a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd12c009ca0 tx=0x7fd12c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 -- 172.21.15.107:0/793645341 shutdown_connections 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 --2- 172.21.15.107:0/793645341 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd13410f2a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 --2- 172.21.15.107:0/793645341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd134104300 0x7fd13410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 --2- 172.21.15.107:0/793645341 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1341039e0 0x7fd134103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 -- 172.21.15.107:0/793645341 >> 172.21.15.107:0/793645341 conn(0x7fd1340fd440 msgr2=0x7fd1340ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.536+0000 7fd138e3a640 1 -- 172.21.15.107:0/793645341 shutdown_connections 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.537+0000 7fd138e3a640 1 -- 172.21.15.107:0/793645341 wait complete. 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.537+0000 7fd138e3a640 1 Processor -- start 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 -- start start 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1341039e0 0x7fd13419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd134104300 0x7fd13419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd134112a10 con 0x7fd13410ce40 2024-08-20T06:34:48.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd134112890 con 0x7fd134104300 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd138e3a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd134112b90 con 0x7fd1341039e0 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58822/0 (socket says 172.21.15.107:58822) 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 -- 172.21.15.107:0/83405673 learned_addr learned my addr 172.21.15.107:0/83405673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd13259c640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1341039e0 0x7fd13419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd129d9b640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd134104300 0x7fd13419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 -- 172.21.15.107:0/83405673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1341039e0 msgr2=0x7fd13419b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1341039e0 0x7fd13419b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 -- 172.21.15.107:0/83405673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd134104300 msgr2=0x7fd13419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd134104300 0x7fd13419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 -- 172.21.15.107:0/83405673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1341a11a0 con 0x7fd13410ce40 2024-08-20T06:34:48.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.538+0000 7fd132d9d640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005e30 tx=0x7fd12c0093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.539+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c032d10 con 0x7fd13410ce40 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.539+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd12c032eb0 con 0x7fd13410ce40 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.539+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1341a1430 con 0x7fd13410ce40 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.539+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c030b20 con 0x7fd13410ce40 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.539+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1341a7b50 con 0x7fd13410ce40 2024-08-20T06:34:48.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.540+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd134103dc0 con 0x7fd13410ce40 2024-08-20T06:34:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.541+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fd12c004040 con 0x7fd13410ce40 2024-08-20T06:34:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.542+0000 7fd12bfff640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 0x7fd1200b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.542+0000 7fd13259c640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 0x7fd1200b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.542+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7fd12c104d70 con 0x7fd13410ce40 2024-08-20T06:34:48.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.542+0000 7fd13259c640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 0x7fd1200b6090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd124009bd0 tx=0x7fd124005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:48.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.545+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd12c0cc840 con 0x7fd13410ce40 2024-08-20T06:34:48.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.758+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1341a7e10 con 0x7fd13410ce40 2024-08-20T06:34:48.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.759+0000 7fd12bfff640 1 -- 172.21.15.107:0/83405673 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9719 (secure 0 0 0) 0x7fd12c0cc140 con 0x7fd13410ce40 2024-08-20T06:34:48.760 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:48.760 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":12,"btime":"2024-08-20T06:34:48:006945+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-08-20T06:34:48.760 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 12 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 msgr2=0x7fd1200b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 0x7fd1200b6090 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd124009bd0 tx=0x7fd124005f90 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 msgr2=0x7fd1341a09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd12c005e30 tx=0x7fd12c0093d0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 shutdown_connections 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd13410ce40 0x7fd1341a09d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd134104300 0x7fd13419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1200b3bd0 0x7fd1200b6090 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 --2- 172.21.15.107:0/83405673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1341039e0 0x7fd13419b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:48.762 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 >> 172.21.15.107:0/83405673 conn(0x7fd1340fd440 msgr2=0x7fd13410f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:48.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 shutdown_connections 2024-08-20T06:34:48.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:48.761+0000 7fd138e3a640 1 -- 172.21.15.107:0/83405673 wait complete. 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: Deploying daemon mds.b on smithi183 2024-08-20T06:34:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.107:6840/1702418500,v1:172.21.15.107:6841/1702418500] up:boot 2024-08-20T06:34:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: fsmap 11 up:standby 2024-08-20T06:34:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T06:34:48.987 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:48 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/83405673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:49.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:49.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: Deploying daemon mds.b on smithi183 2024-08-20T06:34:49.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.107:6840/1702418500,v1:172.21.15.107:6841/1702418500] up:boot 2024-08-20T06:34:49.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: fsmap 11 up:standby 2024-08-20T06:34:49.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T06:34:49.112 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:48 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/83405673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:49.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: Deploying daemon mds.b on smithi183 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.107:6840/1702418500,v1:172.21.15.107:6841/1702418500] up:boot 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: fsmap 11 up:standby 2024-08-20T06:34:49.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T06:34:49.306 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:48 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/83405673' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:49.986 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:49 smithi183 systemd[1]: Starting Ceph mds.b for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:34:50.366 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 podman[51938]: 2024-08-20 06:34:50.063641674 +0000 UTC m=+0.089559420 container create 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-20T06:34:50.366 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 podman[51938]: 2024-08-20 06:34:49.989878421 +0000 UTC m=+0.015796174 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:34:50.366 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 podman[51938]: 2024-08-20 06:34:50.274338759 +0000 UTC m=+0.300256505 container init 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-08-20T06:34:50.366 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 podman[51938]: 2024-08-20 06:34:50.277241236 +0000 UTC m=+0.303158992 container start 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:34:50.367 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mds[51957]: -- 172.21.15.183:0/187335107 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4437b3860 con 0x55a443790800 2024-08-20T06:34:50.367 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: starting mds.b at 2024-08-20T06:34:50.367 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 bash[51938]: 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 2024-08-20T06:34:50.735 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:34:50 smithi183 systemd[1]: Started Ceph mds.b for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:34:51.095 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.096 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:50 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:51.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:51.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.107 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.108 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:50 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:51.116 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:51.117 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:50 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:51.533 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:34:51.827 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:52.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:51 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:boot 2024-08-20T06:34:52.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:51 smithi107 ceph-mon[18495]: fsmap 12 up:standby 2024-08-20T06:34:52.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:34:52.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:51 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:51 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:boot 2024-08-20T06:34:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:51 smithi183 ceph-mon[22204]: fsmap 12 up:standby 2024-08-20T06:34:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:34:52.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:51 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:51 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:boot 2024-08-20T06:34:52.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:51 smithi193 ceph-mon[21944]: fsmap 12 up:standby 2024-08-20T06:34:52.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:34:52.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:51 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 -- 172.21.15.107:0/647627062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c104300 msgr2=0x7f8e4c10c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 --2- 172.21.15.107:0/647627062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c104300 0x7f8e4c10c7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e38009900 tx=0x7f8e3802a410 comp rx=0 tx=0).stop 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 -- 172.21.15.107:0/647627062 shutdown_connections 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 --2- 172.21.15.107:0/647627062 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 --2- 172.21.15.107:0/647627062 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c104300 0x7f8e4c10c7f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 --2- 172.21.15.107:0/647627062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 -- 172.21.15.107:0/647627062 >> 172.21.15.107:0/647627062 conn(0x7f8e4c0fd440 msgr2=0x7f8e4c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:52.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 -- 172.21.15.107:0/647627062 shutdown_connections 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.293+0000 7f8e53009640 1 -- 172.21.15.107:0/647627062 wait complete. 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 Processor -- start 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 -- start start 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c104300 0x7f8e4c101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1129b0 con 0x7f8e4c10ce40 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112830 con 0x7f8e4c104300 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e53009640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112b30 con 0x7f8e4c1039e0 2024-08-20T06:34:52.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e515a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e50da5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e515a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58844/0 (socket says 172.21.15.107:58844) 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e515a6640 1 -- 172.21.15.107:0/571785246 learned_addr learned my addr 172.21.15.107:0/571785246 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.294+0000 7f8e4bfff640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c104300 0x7f8e4c101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 -- 172.21.15.107:0/571785246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c104300 msgr2=0x7f8e4c101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c104300 0x7f8e4c101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 -- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 msgr2=0x7f8e4c102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 -- 172.21.15.107:0/571785246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1a7340 con 0x7f8e4c1039e0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e515a6640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e50da5640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c101690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d720 tx=0x7f8e4000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40017070 con 0x7f8e4c1039e0 2024-08-20T06:34:52.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1a7630 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8e4000de30 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e400047a0 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.295+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a7b40 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.296+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e4c103dc0 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.297+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f8e400099d0 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.298+0000 7f8e49ffb640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 0x7f8e3c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.298+0000 7f8e4bfff640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 0x7f8e3c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.298+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f8e400e3330 con 0x7f8e4c1039e0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.298+0000 7f8e4bfff640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 0x7f8e3c0b6090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e380057c0 tx=0x7f8e38005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:52.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.302+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8e40020020 con 0x7f8e4c1039e0 2024-08-20T06:34:52.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.525+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e4c02e340 con 0x7f8e4c1039e0 2024-08-20T06:34:52.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.526+0000 7f8e49ffb640 1 -- 172.21.15.107:0/571785246 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10548 (secure 0 0 0) 0x7f8e400aae20 con 0x7f8e4c1039e0 2024-08-20T06:34:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:52.528 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":13,"btime":"2024-08-20T06:34:50:830606+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24533,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24541,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24547,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24548,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24562,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24572,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-20T06:34:52.528 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 13 2024-08-20T06:34:52.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 msgr2=0x7f8e3c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:52.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 0x7f8e3c0b6090 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8e380057c0 tx=0x7f8e38005e80 comp rx=0 tx=0).stop 2024-08-20T06:34:52.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 msgr2=0x7f8e4c101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c101690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d720 tx=0x7f8e4000dbe0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 shutdown_connections 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e4c10ce40 0x7f8e4c102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8e4c104300 0x7f8e4c101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8e3c0b3bd0 0x7f8e3c0b6090 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 --2- 172.21.15.107:0/571785246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e4c1039e0 0x7f8e4c101690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.528+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 >> 172.21.15.107:0/571785246 conn(0x7f8e4c0fd440 msgr2=0x7f8e4c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.529+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 shutdown_connections 2024-08-20T06:34:52.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:52.529+0000 7f8e53009640 1 -- 172.21.15.107:0/571785246 wait complete. 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:52.936 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:52.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:52.937 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/571785246' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:53.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:53.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:53.218 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.219 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:53.219 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:52 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/571785246' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:53.225 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:53.225 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.225 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.225 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:34:53.226 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:52 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/571785246' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:34:53.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-20T06:34:54.093 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- 172.21.15.107:0/1963401035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce04104270 msgr2=0x7fce0410c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 --2- 172.21.15.107:0/1963401035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce04104270 0x7fce0410c850 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcdec009ca0 tx=0x7fcdec02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- 172.21.15.107:0/1963401035 shutdown_connections 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 --2- 172.21.15.107:0/1963401035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce0410ce20 0x7fce0410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 --2- 172.21.15.107:0/1963401035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce04104270 0x7fce0410c850 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 --2- 172.21.15.107:0/1963401035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04103870 0x7fce04103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- 172.21.15.107:0/1963401035 >> 172.21.15.107:0/1963401035 conn(0x7fce040fd400 msgr2=0x7fce040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- 172.21.15.107:0/1963401035 shutdown_connections 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- 172.21.15.107:0/1963401035 wait complete. 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 Processor -- start 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.557+0000 7fce0b020640 1 -- start start 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce04103870 0x7fce04198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04104270 0x7fce041993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112960 con 0x7fce0410ce20 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fce041127e0 con 0x7fce04103870 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fce04112ae0 con 0x7fce04104270 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:54.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58854/0 (socket says 172.21.15.107:58854) 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce08dbc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce04103870 0x7fce04198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 -- 172.21.15.107:0/3478436274 learned_addr learned my addr 172.21.15.107:0/3478436274 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce03fff640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04104270 0x7fce041993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 -- 172.21.15.107:0/3478436274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04104270 msgr2=0x7fce041993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04104270 0x7fce041993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 -- 172.21.15.107:0/3478436274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce04103870 msgr2=0x7fce04198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce04103870 0x7fce04198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 -- 172.21.15.107:0/3478436274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0419ef40 con 0x7fce0410ce20 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce095bd640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00e970 tx=0x7fcdfc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc00ccc0 con 0x7fce0410ce20 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fcdfc00ce60 con 0x7fce0410ce20 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0419f230 con 0x7fce0410ce20 2024-08-20T06:34:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.558+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc010430 con 0x7fce0410ce20 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.559+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce041a58f0 con 0x7fce0410ce20 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.560+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce04103c50 con 0x7fce0410ce20 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.561+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fcdfc004f10 con 0x7fce0410ce20 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.561+0000 7fce01ffb640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 0x7fcdf00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.561+0000 7fce08dbc640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 0x7fcdf00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.562+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7fcdfc0e3b20 con 0x7fce0410ce20 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.562+0000 7fce08dbc640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 0x7fcdf00b6090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005fd0 tx=0x7fcdf400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:54.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.565+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fcdfc0ab610 con 0x7fce0410ce20 2024-08-20T06:34:54.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.759+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fce04107900 con 0x7fce0410ce20 2024-08-20T06:34:54.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.759+0000 7fce01ffb640 1 -- 172.21.15.107:0/3478436274 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7fcdfc0055d0 con 0x7fce0410ce20 2024-08-20T06:34:54.760 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout:[ 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout: "poolnum": 1, 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout: "poolname": ".mgr" 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:34:54.761 INFO:teuthology.orchestra.run.smithi107.stdout:] 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 msgr2=0x7fcdf00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 0x7fcdf00b6090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005fd0 tx=0x7fcdf400b390 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 msgr2=0x7fce0419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00e970 tx=0x7fcdfc00ed70 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 shutdown_connections 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fce0410ce20 0x7fce0419e770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcdf00b3bd0 0x7fcdf00b6090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce04104270 0x7fce041993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 --2- 172.21.15.107:0/3478436274 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fce04103870 0x7fce04198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:54.763 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 >> 172.21.15.107:0/3478436274 conn(0x7fce040fd400 msgr2=0x7fce040fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:54.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 shutdown_connections 2024-08-20T06:34:54.764 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:54.762+0000 7fce0b020640 1 -- 172.21.15.107:0/3478436274 wait complete. 2024-08-20T06:34:55.056 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:55.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.057 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:54 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3478436274' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:54 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3478436274' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T06:34:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:55.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:55.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:54 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3478436274' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T06:34:55.371 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-20T06:34:55.371 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-20T06:34:55.642 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:56.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/714323794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18107d20 msgr2=0x7f7a1810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:56.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/714323794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18107d20 0x7f7a1810e520 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7a10009ca0 tx=0x7f7a1002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/714323794 shutdown_connections 2024-08-20T06:34:56.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/714323794 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18107d20 0x7f7a1810e520 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/714323794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 0x7f7a181077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/714323794 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18101990 0x7f7a18101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/714323794 >> 172.21.15.107:0/714323794 conn(0x7f7a180fd400 msgr2=0x7f7a180ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/714323794 shutdown_connections 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.077+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/714323794 wait complete. 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 Processor -- start 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 -- start start 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 0x7f7a180fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18107d20 0x7f7a180fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:56.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18111920 con 0x7f7a18101990 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a181117a0 con 0x7f7a18107d20 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1fc7e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a18111aa0 con 0x7f7a181073a0 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54086/0 (socket says 172.21.15.107:54086) 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 -- 172.21.15.107:0/12322053 learned_addr learned my addr 172.21.15.107:0/12322053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1d219640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 0x7f7a180fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1e21b640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18107d20 0x7f7a180fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 -- 172.21.15.107:0/12322053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 msgr2=0x7f7a180fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 0x7f7a180fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 -- 172.21.15.107:0/12322053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18107d20 msgr2=0x7f7a180fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18107d20 0x7f7a180fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 -- 172.21.15.107:0/12322053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a181a7370 con 0x7f7a18101990 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.078+0000 7f7a1da1a640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400b4a0 tx=0x7f7a1400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:56.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.079+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14015020 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.079+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a14011cc0 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.079+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14010d00 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.079+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a181a7660 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.079+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a181a7bd0 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.080+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a18101d70 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.081+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f7a14018020 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.082+0000 7f7a0affd640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 0x7f7a040b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.082+0000 7f7a1d219640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 0x7f7a040b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.082+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7127+0+0 (secure 0 0 0) 0x7f7a140e2f70 con 0x7f7a18101990 2024-08-20T06:34:56.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.082+0000 7f7a1d219640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 0x7f7a040b6090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ff3f0 tx=0x7f7a0c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:56.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.086+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7a1401f020 con 0x7f7a18101990 2024-08-20T06:34:56.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.262+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7a181a7f20 con 0x7f7a18101990 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:56 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:56 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:34:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:56 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T06:34:56.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.894+0000 7f7a0affd640 1 -- 172.21.15.107:0/12322053 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7f7a140aaa60 con 0x7f7a18101990 2024-08-20T06:34:56.895 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' created 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 msgr2=0x7f7a040b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 0x7f7a040b6090 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7a180ff3f0 tx=0x7f7a0c009110 comp rx=0 tx=0).stop 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 msgr2=0x7f7a180fd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a1400b4a0 tx=0x7f7a1400b8a0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 shutdown_connections 2024-08-20T06:34:56.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a18107d20 0x7f7a180fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a040b3bd0 0x7f7a040b6090 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a181073a0 0x7f7a180fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 --2- 172.21.15.107:0/12322053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a18101990 0x7f7a180fd900 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 >> 172.21.15.107:0/12322053 conn(0x7f7a180fd400 msgr2=0x7f7a181044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 shutdown_connections 2024-08-20T06:34:56.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:56.897+0000 7f7a1fc7e640 1 -- 172.21.15.107:0/12322053 wait complete. 2024-08-20T06:34:57.579 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-08-20T06:34:57.826 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:34:58.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T06:34:58.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:57 smithi183 ceph-mon[22204]: osdmap e67: 12 total, 12 up, 12 in 2024-08-20T06:34:58.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T06:34:58.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:57 smithi107 ceph-mon[18495]: osdmap e67: 12 total, 12 up, 12 in 2024-08-20T06:34:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/12322053' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T06:34:58.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:57 smithi193 ceph-mon[21944]: osdmap e67: 12 total, 12 up, 12 in 2024-08-20T06:34:58.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 -- 172.21.15.107:0/2575222550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1073c0 msgr2=0x7faaec107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:58.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 --2- 172.21.15.107:0/2575222550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1073c0 0x7faaec107800 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7faad4009900 tx=0x7faad402a410 comp rx=0 tx=0).stop 2024-08-20T06:34:58.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 -- 172.21.15.107:0/2575222550 shutdown_connections 2024-08-20T06:34:58.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 --2- 172.21.15.107:0/2575222550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 --2- 172.21.15.107:0/2575222550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1073c0 0x7faaec107800 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 --2- 172.21.15.107:0/2575222550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1019b0 0x7faaec101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 -- 172.21.15.107:0/2575222550 >> 172.21.15.107:0/2575222550 conn(0x7faaec0fd420 msgr2=0x7faaec0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 -- 172.21.15.107:0/2575222550 shutdown_connections 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.337+0000 7faaf136b640 1 -- 172.21.15.107:0/2575222550 wait complete. 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 Processor -- start 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 -- start start 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:58.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1073c0 0x7faaec0fde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec0fe3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7faaec1118b0 con 0x7faaec1019b0 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7faaec111730 con 0x7faaec1073c0 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaf136b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7faaec111a30 con 0x7faaec107d40 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:58.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54112/0 (socket says 172.21.15.107:54112) 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 -- 172.21.15.107:0/751295222 learned_addr learned my addr 172.21.15.107:0/751295222 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faae3fff640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1073c0 0x7faaec0fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeb7fe640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec0fe3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 -- 172.21.15.107:0/751295222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 msgr2=0x7faaec0fe3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec0fe3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 -- 172.21.15.107:0/751295222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1073c0 msgr2=0x7faaec0fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.338+0000 7faaeaffd640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1073c0 0x7faaec0fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faaeaffd640 1 -- 172.21.15.107:0/751295222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaec1a72f0 con 0x7faaec1019b0 2024-08-20T06:34:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faaeb7fe640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec0fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:34:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faaeaffd640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d950 tx=0x7faadc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:58.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc017070 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7faadc009c30 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaec1a75e0 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faadc00f620 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.339+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaec1a7af0 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.340+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaec101d90 con 0x7faaec1019b0 2024-08-20T06:34:58.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.341+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7faadc010460 con 0x7faaec1019b0 2024-08-20T06:34:58.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.342+0000 7faae8ff9640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 0x7faad80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:34:58.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.342+0000 7faae3fff640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 0x7faad80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:34:58.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.342+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7497+0+0 (secure 0 0 0) 0x7faadc0e3540 con 0x7faaec1019b0 2024-08-20T06:34:58.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.342+0000 7faae3fff640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 0x7faad80b6090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faad40057c0 tx=0x7faad40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:34:58.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.345+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7faadc0aaec0 con 0x7faaec1019b0 2024-08-20T06:34:58.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.524+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faaec1a7e60 con 0x7faaec1019b0 2024-08-20T06:34:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.906+0000 7faae8ff9640 1 -- 172.21.15.107:0/751295222 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v69) ==== 170+0+0 (secure 0 0 0) 0x7faadc009de0 con 0x7faaec1019b0 2024-08-20T06:34:58.907 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' created 2024-08-20T06:34:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 msgr2=0x7faad80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 0x7faad80b6090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7faad40057c0 tx=0x7faad40058d0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 msgr2=0x7faaec0fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:34:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7faadc00d950 tx=0x7faadc00de10 comp rx=0 tx=0).stop 2024-08-20T06:34:58.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 shutdown_connections 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7faad80b3bd0 0x7faad80b6090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faaec107d40 0x7faaec0fe3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7faaec1073c0 0x7faaec0fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 --2- 172.21.15.107:0/751295222 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7faaec1019b0 0x7faaec0fd920 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 >> 172.21.15.107:0/751295222 conn(0x7faaec0fd420 msgr2=0x7faaec06b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 shutdown_connections 2024-08-20T06:34:58.910 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:34:58.908+0000 7faaf136b640 1 -- 172.21.15.107:0/751295222 wait complete. 2024-08-20T06:34:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:58 smithi107 ceph-mon[18495]: pgmap v265: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:58 smithi107 ceph-mon[18495]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T06:34:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:58 smithi107 ceph-mon[18495]: osdmap e68: 12 total, 12 up, 12 in 2024-08-20T06:34:59.231 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T06:34:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:58 smithi183 ceph-mon[22204]: pgmap v265: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:58 smithi183 ceph-mon[22204]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T06:34:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:58 smithi183 ceph-mon[22204]: osdmap e68: 12 total, 12 up, 12 in 2024-08-20T06:34:59.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T06:34:59.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:58 smithi193 ceph-mon[21944]: pgmap v265: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:34:59.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:58 smithi193 ceph-mon[21944]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T06:34:59.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:58 smithi193 ceph-mon[21944]: osdmap e68: 12 total, 12 up, 12 in 2024-08-20T06:34:59.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T06:34:59.559 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-20T06:34:59.802 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:00.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:59 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T06:35:00.178 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:34:59 smithi107 ceph-mon[18495]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T06:35:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:59 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T06:35:00.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:34:59 smithi183 ceph-mon[22204]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T06:35:00.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 -- 172.21.15.107:0/1231210163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 msgr2=0x7f09b41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:00.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/1231210163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41077e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009900 tx=0x7f09a002a410 comp rx=0 tx=0).stop 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 -- 172.21.15.107:0/1231210163 shutdown_connections 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/1231210163 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/1231210163 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41077e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/1231210163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.259+0000 7f09bb0a6640 1 -- 172.21.15.107:0/1231210163 >> 172.21.15.107:0/1231210163 conn(0x7f09b40fd400 msgr2=0x7f09b40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- 172.21.15.107:0/1231210163 shutdown_connections 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- 172.21.15.107:0/1231210163 wait complete. 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 Processor -- start 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- start start 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4111920 con 0x7f09b41073a0 2024-08-20T06:35:00.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f09b41117a0 con 0x7f09b4107d20 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09bb0a6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f09b4111aa0 con 0x7f09b4101990 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09b3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09b3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54128/0 (socket says 172.21.15.107:54128) 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.260+0000 7f09b3fff640 1 -- 172.21.15.107:0/697803055 learned_addr learned my addr 172.21.15.107:0/697803055 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b9643640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b8e42640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 -- 172.21.15.107:0/697803055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 msgr2=0x7f09b4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 -- 172.21.15.107:0/697803055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 msgr2=0x7f09b419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 -- 172.21.15.107:0/697803055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b419e040 con 0x7f09b41073a0 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b8e42640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b9643640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:00.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b3fff640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09a002a930 tx=0x7f09a0003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a003f070 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b419e2d0 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f09a003bdc0 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0035030 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.261+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b41a49f0 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.262+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b4101df0 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.264+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f09a0042070 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.265+0000 7f09b1ffb640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 0x7f09a40b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.265+0000 7f09b8e42640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 0x7f09a40b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.265+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7879+0+0 (secure 0 0 0) 0x7f09a0104980 con 0x7f09b41073a0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.265+0000 7f09b8e42640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 0x7f09a40b6090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09a8006fd0 tx=0x7f09a8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:00.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.268+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f09a00cc1f0 con 0x7f09b41073a0 2024-08-20T06:35:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:59 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/751295222' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T06:35:00.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:34:59 smithi193 ceph-mon[21944]: osdmap e69: 12 total, 12 up, 12 in 2024-08-20T06:35:00.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.477+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f09b41990c0 con 0x7f09b41073a0 2024-08-20T06:35:00.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.913+0000 7f09b1ffb640 1 -- 172.21.15.107:0/697803055 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f09b41990c0 con 0x7f09b41073a0 2024-08-20T06:35:00.914 INFO:teuthology.orchestra.run.smithi107.stderr:new fs with metadata pool 2 and data pool 3 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 msgr2=0x7f09a40b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 0x7f09a40b6090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f09a8006fd0 tx=0x7f09a8008040 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 msgr2=0x7f09b41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f09a002a930 tx=0x7f09a0003d60 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 shutdown_connections 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f09b4107d20 0x7f09b419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f09b41073a0 0x7f09b41984d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f09a40b3bd0 0x7f09a40b6090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 --2- 172.21.15.107:0/697803055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b4101990 0x7f09b4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:00.917 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 >> 172.21.15.107:0/697803055 conn(0x7f09b40fd400 msgr2=0x7f09b40ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:00.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 shutdown_connections 2024-08-20T06:35:00.918 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:00.916+0000 7f09bb0a6640 1 -- 172.21.15.107:0/697803055 wait complete. 2024-08-20T06:35:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:00 smithi183 ceph-mon[22204]: pgmap v268: 97 pgs: 1 creating+activating, 11 creating+peering, 3 active+clean, 82 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:35:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:00 smithi183 ceph-mon[22204]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T06:35:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:00 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T06:35:01.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:00 smithi183 ceph-mon[22204]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T06:35:01.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:00 smithi107 ceph-mon[18495]: pgmap v268: 97 pgs: 1 creating+activating, 11 creating+peering, 3 active+clean, 82 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:35:01.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:00 smithi107 ceph-mon[18495]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T06:35:01.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:00 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T06:35:01.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:00 smithi107 ceph-mon[18495]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T06:35:01.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:00 smithi193 ceph-mon[21944]: pgmap v268: 97 pgs: 1 creating+activating, 11 creating+peering, 3 active+clean, 82 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:35:01.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:00 smithi193 ceph-mon[21944]: osdmap e70: 12 total, 12 up, 12 in 2024-08-20T06:35:01.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:00 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T06:35:01.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:00 smithi193 ceph-mon[21944]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T06:35:01.646 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:35:01.894 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:02.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:01 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T06:35:02.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:01 smithi107 ceph-mon[18495]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T06:35:02.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:01 smithi107 ceph-mon[18495]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T06:35:02.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:01 smithi107 ceph-mon[18495]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T06:35:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:01 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T06:35:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:01 smithi183 ceph-mon[22204]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T06:35:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:01 smithi183 ceph-mon[22204]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T06:35:02.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:01 smithi183 ceph-mon[22204]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T06:35:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:01 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/697803055' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T06:35:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:01 smithi193 ceph-mon[21944]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T06:35:02.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:01 smithi193 ceph-mon[21944]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T06:35:02.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:01 smithi193 ceph-mon[21944]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T06:35:02.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 -- 172.21.15.107:0/1685735367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 msgr2=0x7fdb3010f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:02.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1685735367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3010f680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009ca0 tx=0x7fdb2802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 -- 172.21.15.107:0/1685735367 shutdown_connections 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1685735367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3010f680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1685735367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1685735367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 -- 172.21.15.107:0/1685735367 >> 172.21.15.107:0/1685735367 conn(0x7fdb300fe540 msgr2=0x7fdb30100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:02.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 -- 172.21.15.107:0/1685735367 shutdown_connections 2024-08-20T06:35:02.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.356+0000 7fdb351bb640 1 -- 172.21.15.107:0/1685735367 wait complete. 2024-08-20T06:35:02.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.357+0000 7fdb351bb640 1 Processor -- start 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 -- start start 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30112a80 con 0x7fdb30108e80 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30112900 con 0x7fdb30108500 2024-08-20T06:35:02.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb351bb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb30112c00 con 0x7fdb30102af0 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54142/0 (socket says 172.21.15.107:54142) 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 -- 172.21.15.107:0/1548499562 learned_addr learned my addr 172.21.15.107:0/1548499562 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2659c640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2ed9d640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 -- 172.21.15.107:0/1548499562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 msgr2=0x7fdb30198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 -- 172.21.15.107:0/1548499562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 msgr2=0x7fdb30199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:02.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 -- 172.21.15.107:0/1548499562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3019efd0 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2ed9d640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2659c640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2f59e640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb28031040 tx=0x7fdb28032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.358+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28004040 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.359+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb28004670 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.359+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3019f260 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.359+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28030d50 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.359+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301a5a00 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.360+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb30102ed0 con 0x7fdb30108e80 2024-08-20T06:35:02.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.361+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fdb280305c0 con 0x7fdb30108e80 2024-08-20T06:35:02.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.362+0000 7fdb2cd99640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 0x7fdb1c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:02.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.362+0000 7fdb2ed9d640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 0x7fdb1c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:02.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.362+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fdb28105350 con 0x7fdb30108e80 2024-08-20T06:35:02.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.362+0000 7fdb2ed9d640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 0x7fdb1c0b6090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:02.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.365+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdb280ccaf0 con 0x7fdb30108e80 2024-08-20T06:35:02.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.539+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb3019a050 con 0x7fdb30108e80 2024-08-20T06:35:02.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.541+0000 7fdb2cd99640 1 -- 172.21.15.107:0/1548499562 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+22371 (secure 0 0 0) 0x7fdb280cc3f0 con 0x7fdb30108e80 2024-08-20T06:35:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:02.543 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":71,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:35:00.478609+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 msgr2=0x7fdb1c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 0x7fdb1c0b6090 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb20006fd0 tx=0x7fdb20008040 comp rx=0 tx=0).stop 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 msgr2=0x7fdb3019e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fdb28031040 tx=0x7fdb28032bc0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 shutdown_connections 2024-08-20T06:35:02.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb30108e80 0x7fdb3019e800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb30108500 0x7fdb30199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb1c0b3bd0 0x7fdb1c0b6090 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 --2- 172.21.15.107:0/1548499562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb30102af0 0x7fdb30198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 >> 172.21.15.107:0/1548499562 conn(0x7fdb300fe540 msgr2=0x7fdb30105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 shutdown_connections 2024-08-20T06:35:02.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:02.543+0000 7fdb351bb640 1 -- 172.21.15.107:0/1548499562 wait complete. 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: pgmap v271: 97 pgs: 5 creating+activating, 11 creating+peering, 38 active+clean, 43 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: Cluster is now healthy 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:active 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.233 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:02 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1548499562' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: pgmap v271: 97 pgs: 5 creating+activating, 11 creating+peering, 38 active+clean, 43 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: Cluster is now healthy 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:active 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:02 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1548499562' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:03.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:35:03.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: pgmap v271: 97 pgs: 5 creating+activating, 11 creating+peering, 38 active+clean, 43 unknown; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 4 op/s 2024-08-20T06:35:03.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T06:35:03.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: Cluster is now healthy 2024-08-20T06:35:03.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] up:active 2024-08-20T06:35:03.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:03.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:02 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1548499562' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:03.502 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 -- 172.21.15.107:0/1032146151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1073d0 msgr2=0x7f31ac107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/1032146151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1073d0 0x7f31ac107810 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3198009900 tx=0x7f319802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 -- 172.21.15.107:0/1032146151 shutdown_connections 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/1032146151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/1032146151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1073d0 0x7f31ac107810 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/1032146151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1019c0 0x7f31ac101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 -- 172.21.15.107:0/1032146151 >> 172.21.15.107:0/1032146151 conn(0x7f31ac0fd410 msgr2=0x7f31ac0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.996+0000 7f31b2ef8640 1 -- 172.21.15.107:0/1032146151 shutdown_connections 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 -- 172.21.15.107:0/1032146151 wait complete. 2024-08-20T06:35:04.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 Processor -- start 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 -- start start 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 0x7f31ac198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac111910 con 0x7f31ac1019c0 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac111790 con 0x7f31ac107d50 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b2ef8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac111a90 con 0x7f31ac1073d0 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.997+0000 7f31b0c94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54174/0 (socket says 172.21.15.107:54174) 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 -- 172.21.15.107:0/3007000897 learned_addr learned my addr 172.21.15.107:0/3007000897 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b1495640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:04.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31abfff640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 0x7f31ac198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 -- 172.21.15.107:0/3007000897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 msgr2=0x7f31ac198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 0x7f31ac198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 -- 172.21.15.107:0/3007000897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 msgr2=0x7f31ac19d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac19d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 -- 172.21.15.107:0/3007000897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac19e0e0 con 0x7f31ac1019c0 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b1495640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31abfff640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 0x7f31ac198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b0c94640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c890 tx=0x7f31a000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0004220 con 0x7f31ac1019c0 2024-08-20T06:35:04.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f31a0007e40 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac19e3d0 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0004a00 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.998+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac06b1b0 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:03.999+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31ac101da0 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.000+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f31a00079b0 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.001+0000 7f31a9ffb640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 0x7f319c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.001+0000 7f31abfff640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 0x7f319c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.001+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f31a00e38a0 con 0x7f31ac1019c0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.002+0000 7f31abfff640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 0x7f319c0b6090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f319802a930 tx=0x7f3198005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:04.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.005+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f31a0014030 con 0x7f31ac1019c0 2024-08-20T06:35:04.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.181+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31ac199160 con 0x7f31ac1019c0 2024-08-20T06:35:04.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.182+0000 7f31a9ffb640 1 -- 172.21.15.107:0/3007000897 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+22371 (secure 0 0 0) 0x7f31a00ab040 con 0x7f31ac1019c0 2024-08-20T06:35:04.183 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:04.184 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":71,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:35:00.478609+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:35:04.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.184+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 msgr2=0x7f319c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:04.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.184+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 0x7f319c0b6090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f319802a930 tx=0x7f3198005fd0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.184+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 msgr2=0x7f31ac198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.184+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f31a000c890 tx=0x7f31a000cd50 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 shutdown_connections 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f31ac107d50 0x7f31ac19d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f319c0b3bd0 0x7f319c0b6090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31ac1073d0 0x7f31ac198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 --2- 172.21.15.107:0/3007000897 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f31ac1019c0 0x7f31ac198030 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 >> 172.21.15.107:0/3007000897 conn(0x7f31ac0fd410 msgr2=0x7f31ac0ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 shutdown_connections 2024-08-20T06:35:04.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:04.185+0000 7f31b2ef8640 1 -- 172.21.15.107:0/3007000897 wait complete. 2024-08-20T06:35:04.836 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-20T06:35:05.081 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:05.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:04 smithi107 ceph-mon[18495]: pgmap v272: 97 pgs: 4 creating+activating, 93 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 760 B/s wr, 5 op/s 2024-08-20T06:35:05.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:04 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3007000897' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:04 smithi183 ceph-mon[22204]: pgmap v272: 97 pgs: 4 creating+activating, 93 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 760 B/s wr, 5 op/s 2024-08-20T06:35:05.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:04 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3007000897' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:05.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:04 smithi193 ceph-mon[21944]: pgmap v272: 97 pgs: 4 creating+activating, 93 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 760 B/s wr, 5 op/s 2024-08-20T06:35:05.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:04 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3007000897' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 -- 172.21.15.107:0/4107287583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3268107d40 msgr2=0x7f326810e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 --2- 172.21.15.107:0/4107287583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3268107d40 0x7f326810e540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3260009ca0 tx=0x7f326002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 -- 172.21.15.107:0/4107287583 shutdown_connections 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 --2- 172.21.15.107:0/4107287583 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3268107d40 0x7f326810e540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 --2- 172.21.15.107:0/4107287583 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 0x7f3268107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 --2- 172.21.15.107:0/4107287583 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f32681019b0 0x7f3268101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 -- 172.21.15.107:0/4107287583 >> 172.21.15.107:0/4107287583 conn(0x7f32680fd420 msgr2=0x7f32680ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 -- 172.21.15.107:0/4107287583 shutdown_connections 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.551+0000 7f326f169640 1 -- 172.21.15.107:0/4107287583 wait complete. 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 Processor -- start 2024-08-20T06:35:05.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 -- start start 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 0x7f32681983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3268107d40 0x7f326819d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3268111940 con 0x7f32681019b0 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f32681117c0 con 0x7f3268107d40 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.552+0000 7f326f169640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3268111ac0 con 0x7f32681073c0 2024-08-20T06:35:05.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59890/0 (socket says 172.21.15.107:59890) 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 -- 172.21.15.107:0/501562665 learned_addr learned my addr 172.21.15.107:0/501562665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326d706640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3268107d40 0x7f326819d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f3267fff640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 0x7f32681983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 -- 172.21.15.107:0/501562665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 msgr2=0x7f32681983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 0x7f32681983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 -- 172.21.15.107:0/501562665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3268107d40 msgr2=0x7f326819d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3268107d40 0x7f326819d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 -- 172.21.15.107:0/501562665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f326819df10 con 0x7f32681019b0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326cf05640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f325c00b4a0 tx=0x7f325c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c014070 con 0x7f32681019b0 2024-08-20T06:35:05.557 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f325c010aa0 con 0x7f32681019b0 2024-08-20T06:35:05.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f326819e200 con 0x7f32681019b0 2024-08-20T06:35:05.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.553+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325c017b70 con 0x7f32681019b0 2024-08-20T06:35:05.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.554+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32681a4940 con 0x7f32681019b0 2024-08-20T06:35:05.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.554+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3268101d90 con 0x7f32681019b0 2024-08-20T06:35:05.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.556+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f325c017070 con 0x7f32681019b0 2024-08-20T06:35:05.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.556+0000 7f3265ffb640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 0x7f32580b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:05.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.557+0000 7f3267fff640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 0x7f32580b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:05.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.557+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f325c0e3240 con 0x7f32681019b0 2024-08-20T06:35:05.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.557+0000 7f3267fff640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 0x7f32580b6090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3268103970 tx=0x7f3254008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:05.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.560+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f325c0aa9e0 con 0x7f32681019b0 2024-08-20T06:35:05.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.774+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f326802e340 con 0x7f32681019b0 2024-08-20T06:35:05.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.937+0000 7f3265ffb640 1 -- 172.21.15.107:0/501562665 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f325c0aa2e0 con 0x7f32681019b0 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 msgr2=0x7f32580b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 0x7f32580b6090 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3268103970 tx=0x7f3254008040 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 msgr2=0x7f3268197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f325c00b4a0 tx=0x7f325c00b960 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 shutdown_connections 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3268107d40 0x7f326819d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32580b3bd0 0x7f32580b6090 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32681073c0 0x7f32681983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 --2- 172.21.15.107:0/501562665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32681019b0 0x7f3268197e60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.939+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 >> 172.21.15.107:0/501562665 conn(0x7f32680fd420 msgr2=0x7f32680ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.940+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 shutdown_connections 2024-08-20T06:35:05.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:05.940+0000 7f326f169640 1 -- 172.21.15.107:0/501562665 wait complete. 2024-08-20T06:35:06.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T06:35:06.236 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:35:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: 2024-08-20T06:35:05.841+0000 7ff1ce1f7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-20T06:35:06.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T06:35:06.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T06:35:06.637 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T06:35:06.637 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-20T06:35:06.883 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:06 smithi183 ceph-mon[22204]: pgmap v273: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T06:35:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:06 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T06:35:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:06 smithi183 ceph-mon[22204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:06 smithi107 ceph-mon[18495]: pgmap v273: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T06:35:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:06 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T06:35:07.254 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:06 smithi107 ceph-mon[18495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:06 smithi193 ceph-mon[21944]: pgmap v273: 97 pgs: 97 active+clean; 578 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-08-20T06:35:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:06 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/501562665' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T06:35:07.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:06 smithi193 ceph-mon[21944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:07.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 -- 172.21.15.107:0/693196824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00701073a0 msgr2=0x7f00701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 --2- 172.21.15.107:0/693196824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00701073a0 0x7f00701077e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0060009900 tx=0x7f006002a410 comp rx=0 tx=0).stop 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 -- 172.21.15.107:0/693196824 shutdown_connections 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 --2- 172.21.15.107:0/693196824 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070107d20 0x7f007010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 --2- 172.21.15.107:0/693196824 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00701073a0 0x7f00701077e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 --2- 172.21.15.107:0/693196824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0070101990 0x7f0070101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 -- 172.21.15.107:0/693196824 >> 172.21.15.107:0/693196824 conn(0x7f00700fd400 msgr2=0x7f00700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 -- 172.21.15.107:0/693196824 shutdown_connections 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 -- 172.21.15.107:0/693196824 wait complete. 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.338+0000 7f0077d42640 1 Processor -- start 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 -- start start 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101990 0x7f0070199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:07.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00701073a0 0x7f00701995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00701118d0 con 0x7f0070107d20 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0070111750 con 0x7f0070101990 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0077d42640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0070111a50 con 0x7f00701073a0 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59904/0 (socket says 172.21.15.107:59904) 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 -- 172.21.15.107:0/3106610759 learned_addr learned my addr 172.21.15.107:0/3106610759 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f0075ade640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101990 0x7f0070199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00752dd640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00701073a0 0x7f00701995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 -- 172.21.15.107:0/3106610759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00701073a0 msgr2=0x7f00701995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00701073a0 0x7f00701995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 -- 172.21.15.107:0/3106610759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101990 msgr2=0x7f0070199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:07.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101990 0x7f0070199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 -- 172.21.15.107:0/3106610759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f007019f110 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.339+0000 7f00762df640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f006800b970 tx=0x7f006800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.340+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0068019070 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.340+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0068014440 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.340+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f007019f400 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.340+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00680075d0 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.340+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00701a5b40 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.341+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0070101df0 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.342+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f006801c070 con 0x7f0070107d20 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.343+0000 7f0066ffd640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 0x7f005c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.343+0000 7f0075ade640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 0x7f005c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:07.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.343+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f00680e33a0 con 0x7f0070107d20 2024-08-20T06:35:07.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.344+0000 7f0075ade640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 0x7f005c0b6110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f006c009990 tx=0x7f006c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:07.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.346+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f00680aa4f0 con 0x7f0070107d20 2024-08-20T06:35:07.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.563+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f00701a5f30 con 0x7f0070107d20 2024-08-20T06:35:07.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.946+0000 7f0066ffd640 1 -- 172.21.15.107:0/3106610759 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f00680aea40 con 0x7f0070107d20 2024-08-20T06:35:07.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 msgr2=0x7f005c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:07.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 0x7f005c0b6110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f006c009990 tx=0x7f006c009110 comp rx=0 tx=0).stop 2024-08-20T06:35:07.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 msgr2=0x7f007019e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:07.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f006800b970 tx=0x7f006800bd70 comp rx=0 tx=0).stop 2024-08-20T06:35:07.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 shutdown_connections 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0070107d20 0x7f007019e940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f005c0b3c50 0x7f005c0b6110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00701073a0 0x7f00701995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 --2- 172.21.15.107:0/3106610759 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0070101990 0x7f0070199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 >> 172.21.15.107:0/3106610759 conn(0x7f00700fd400 msgr2=0x7f0070100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 shutdown_connections 2024-08-20T06:35:07.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:07.949+0000 7f0077d42640 1 -- 172.21.15.107:0/3106610759 wait complete. 2024-08-20T06:35:08.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:07 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T06:35:08.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:07 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T06:35:08.324 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:07 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T06:35:08.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-20T06:35:08.893 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:09.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T06:35:09.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T06:35:09.201 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T06:35:09.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T06:35:09.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:09.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-20T06:35:09.202 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:08 smithi107 ceph-mon[18495]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-20T06:35:09.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:08 smithi183 ceph-mon[22204]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-20T06:35:09.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: pgmap v274: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3106610759' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-08-20T06:35:09.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:08 smithi193 ceph-mon[21944]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 -- 172.21.15.107:0/1903215087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 msgr2=0x7f6d701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1903215087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d701077e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009900 tx=0x7f6d5802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 -- 172.21.15.107:0/1903215087 shutdown_connections 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1903215087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 0x7f6d7010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1903215087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d701077e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1903215087 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 0x7f6d70101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 -- 172.21.15.107:0/1903215087 >> 172.21.15.107:0/1903215087 conn(0x7f6d700fd400 msgr2=0x7f6d700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 -- 172.21.15.107:0/1903215087 shutdown_connections 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.404+0000 7f6d758a7640 1 -- 172.21.15.107:0/1903215087 wait complete. 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 Processor -- start 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 -- start start 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 0x7f6d70197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 0x7f6d7019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70111890 con 0x7f6d701073a0 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70111710 con 0x7f6d70101990 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d758a7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6d70111a10 con 0x7f6d70107d20 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59920/0 (socket says 172.21.15.107:59920) 2024-08-20T06:35:09.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 -- 172.21.15.107:0/1567332228 learned_addr learned my addr 172.21.15.107:0/1567332228 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d6effd640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 0x7f6d70197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d6f7fe640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 0x7f6d7019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 -- 172.21.15.107:0/1567332228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 msgr2=0x7f6d7019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 0x7f6d7019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 -- 172.21.15.107:0/1567332228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 msgr2=0x7f6d70197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 0x7f6d70197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.405+0000 7f6d67fff640 1 -- 172.21.15.107:0/1567332228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7019de90 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d67fff640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009a20 tx=0x7f6d58004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5803f030 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6d580035f0 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7019e120 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58035070 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.406+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d701a4840 con 0x7f6d701073a0 2024-08-20T06:35:09.432 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.407+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d70101d70 con 0x7f6d701073a0 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.408+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f6d58003aa0 con 0x7f6d701073a0 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.410+0000 7f6d748a5640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 0x7f6d5c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.410+0000 7f6d6effd640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 0x7f6d5c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.410+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f6d58104930 con 0x7f6d701073a0 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.410+0000 7f6d6effd640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 0x7f6d5c0b6090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000abe0 tx=0x7f6d6000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:09.433 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.413+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6d580cc0d0 con 0x7f6d701073a0 2024-08-20T06:35:09.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.625+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f6d70198f10 con 0x7f6d701073a0 2024-08-20T06:35:09.969 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.968+0000 7f6d748a5640 1 -- 172.21.15.107:0/1567332228 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f6d70198f10 con 0x7f6d701073a0 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 msgr2=0x7f6d5c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 0x7f6d5c0b6090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6d6000abe0 tx=0x7f6d6000a2d0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 msgr2=0x7f6d70198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d58009a20 tx=0x7f6d58004270 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 shutdown_connections 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6d5c0b3bd0 0x7f6d5c0b6090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d70107d20 0x7f6d7019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d701073a0 0x7f6d70198320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 --2- 172.21.15.107:0/1567332228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6d70101990 0x7f6d70197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 >> 172.21.15.107:0/1567332228 conn(0x7f6d700fd400 msgr2=0x7f6d700ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 shutdown_connections 2024-08-20T06:35:09.972 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:09.971+0000 7f6d758a7640 1 -- 172.21.15.107:0/1567332228 wait complete. 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:active 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: Cluster is now healthy 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T06:35:10.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:09 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:active 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: Cluster is now healthy 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T06:35:10.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:09 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T06:35:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] up:active 2024-08-20T06:35:10.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: Cluster is now healthy 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T06:35:10.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:09 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T06:35:10.638 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-20T06:35:10.883 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: 2024-08-20T06:35:10.842+0000 7ff1ce1f7640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: 2024-08-20T06:35:10.842+0000 7ff1ce1f7640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-mon[22204]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:active 2024-08-20T06:35:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-20T06:35:11.237 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:10 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:11.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:10 smithi107 ceph-mon[18495]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-08-20T06:35:11.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:10 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T06:35:11.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:10 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:active 2024-08-20T06:35:11.245 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:10 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-20T06:35:11.246 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:10 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:10 smithi193 ceph-mon[21944]: pgmap v275: 97 pgs: 97 active+clean; 579 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 5 op/s 2024-08-20T06:35:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:10 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1567332228' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T06:35:11.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:10 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] up:active 2024-08-20T06:35:11.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:10 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-08-20T06:35:11.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:10 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:11.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 -- 172.21.15.107:0/4121210620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c102ab0 msgr2=0x7f5e5c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 --2- 172.21.15.107:0/4121210620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c102e90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009900 tx=0x7f5e4c02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 -- 172.21.15.107:0/4121210620 shutdown_connections 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 --2- 172.21.15.107:0/4121210620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c108e40 0x7f5e5c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 --2- 172.21.15.107:0/4121210620 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c1084c0 0x7f5e5c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 --2- 172.21.15.107:0/4121210620 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c102e90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 -- 172.21.15.107:0/4121210620 >> 172.21.15.107:0/4121210620 conn(0x7f5e5c0fe500 msgr2=0x7f5e5c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 -- 172.21.15.107:0/4121210620 shutdown_connections 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.327+0000 7f5e61e05640 1 -- 172.21.15.107:0/4121210620 wait complete. 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 Processor -- start 2024-08-20T06:35:11.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 -- start start 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c1084c0 0x7f5e5c1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c1129b0 con 0x7f5e5c108e40 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c112830 con 0x7f5e5c102ab0 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e61e05640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c112b30 con 0x7f5e5c1084c0 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e5bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59942/0 (socket says 172.21.15.107:59942) 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e5bfff640 1 -- 172.21.15.107:0/625838547 learned_addr learned my addr 172.21.15.107:0/625838547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e5b7fe640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.328+0000 7f5e5affd640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c1084c0 0x7f5e5c1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:11.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 -- 172.21.15.107:0/625838547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c1084c0 msgr2=0x7f5e5c1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c1084c0 0x7f5e5c1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 -- 172.21.15.107:0/625838547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c102ab0 msgr2=0x7f5e5c198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 -- 172.21.15.107:0/625838547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c19ef50 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e5bfff640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400eaa0 tx=0x7f5e5400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5400cdb0 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e54005020 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e540116d0 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c19f240 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.329+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c1a5900 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.330+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5c102e90 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.331+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f5e540051d0 con 0x7f5e5c108e40 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.333+0000 7f5e58ff9640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 0x7f5e440b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:11.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.333+0000 7f5e5b7fe640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 0x7f5e440b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.333+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f5e540e3fe0 con 0x7f5e5c108e40 2024-08-20T06:35:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.333+0000 7f5e5b7fe640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 0x7f5e440b6090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0057c0 tx=0x7f5e4c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:11.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.336+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5e54016030 con 0x7f5e5c108e40 2024-08-20T06:35:11.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.547+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f5e5c199fd0 con 0x7f5e5c108e40 2024-08-20T06:35:11.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.975+0000 7f5e58ff9640 1 -- 172.21.15.107:0/625838547 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f5e5c199fd0 con 0x7f5e5c108e40 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 msgr2=0x7f5e440b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 0x7f5e440b6090 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0057c0 tx=0x7f5e4c0058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 msgr2=0x7f5e5c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400eaa0 tx=0x7f5e5400ef60 comp rx=0 tx=0).stop 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 shutdown_connections 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e5c108e40 0x7f5e5c19e780 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.979 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e440b3bd0 0x7f5e440b6090 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e5c1084c0 0x7f5e5c1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 --2- 172.21.15.107:0/625838547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e5c102ab0 0x7f5e5c198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:11.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 >> 172.21.15.107:0/625838547 conn(0x7f5e5c0fe500 msgr2=0x7f5e5c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:11.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 shutdown_connections 2024-08-20T06:35:11.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:11.978+0000 7f5e61e05640 1 -- 172.21.15.107:0/625838547 wait complete. 2024-08-20T06:35:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:11 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T06:35:12.274 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:11 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T06:35:12.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:11 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T06:35:12.586 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-08-20T06:35:12.587 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-20T06:35:12.830 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:12 smithi183 ceph-mon[22204]: pgmap v276: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-08-20T06:35:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:12 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T06:35:13.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:12 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:12 smithi107 ceph-mon[18495]: pgmap v276: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-08-20T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:12 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T06:35:13.237 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:12 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:12 smithi193 ceph-mon[21944]: pgmap v276: 97 pgs: 97 active+clean; 581 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 986 B/s rd, 3.2 KiB/s wr, 10 op/s 2024-08-20T06:35:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:12 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/625838547' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T06:35:13.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:12 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- 172.21.15.107:0/471661448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841073d0 msgr2=0x7f9584107810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 --2- 172.21.15.107:0/471661448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841073d0 0x7f9584107810 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f9574009900 tx=0x7f957402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- 172.21.15.107:0/471661448 shutdown_connections 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 --2- 172.21.15.107:0/471661448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 0x7f958410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 --2- 172.21.15.107:0/471661448 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841073d0 0x7f9584107810 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 --2- 172.21.15.107:0/471661448 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841019c0 0x7f9584101da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- 172.21.15.107:0/471661448 >> 172.21.15.107:0/471661448 conn(0x7f95840fd410 msgr2=0x7f95840ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- 172.21.15.107:0/471661448 shutdown_connections 2024-08-20T06:35:13.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- 172.21.15.107:0/471661448 wait complete. 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 Processor -- start 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 -- start start 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.318+0000 7f958b66e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841073d0 0x7f9584198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 0x7f958419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95841118c0 con 0x7f95841019c0 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9584111740 con 0x7f95841073d0 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9584111a40 con 0x7f9584107d50 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59962/0 (socket says 172.21.15.107:59962) 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 -- 172.21.15.107:0/3971521851 learned_addr learned my addr 172.21.15.107:0/3971521851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:13.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f9588c09640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841073d0 0x7f9584198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f9589c0b640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 0x7f958419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 -- 172.21.15.107:0/3971521851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 msgr2=0x7f958419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 0x7f958419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 -- 172.21.15.107:0/3971521851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841073d0 msgr2=0x7f9584198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841073d0 0x7f9584198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 -- 172.21.15.107:0/3971521851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958419e1f0 con 0x7f95841019c0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958940a640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f958000b4a0 tx=0x7f958000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580014070 con 0x7f95841019c0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958419e4e0 con 0x7f95841019c0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9580010aa0 con 0x7f95841019c0 2024-08-20T06:35:13.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580017b70 con 0x7f95841019c0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.319+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95841a4d70 con 0x7f95841019c0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.320+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9584101da0 con 0x7f95841019c0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.323+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7f9580017070 con 0x7f95841019c0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.323+0000 7f957a7fc640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 0x7f95700b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.323+0000 7f9588c09640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 0x7f95700b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.323+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f95800e3150 con 0x7f95841019c0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.324+0000 7f9588c09640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 0x7f95700b6090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95740057c0 tx=0x7f95740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:13.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.326+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f95800aa2a0 con 0x7f95841019c0 2024-08-20T06:35:13.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.530+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f958406ae90 con 0x7f95700b3bd0 2024-08-20T06:35:13.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.549+0000 7f957a7fc640 1 -- 172.21.15.107:0/3971521851 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f958406ae90 con 0x7f95700b3bd0 2024-08-20T06:35:13.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 msgr2=0x7f95700b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:13.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 0x7f95700b6090 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f95740057c0 tx=0x7f95740058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 msgr2=0x7f9584198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f958000b4a0 tx=0x7f958000b960 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 shutdown_connections 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f95700b3bd0 0x7f95700b6090 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9584107d50 0x7f958419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95841073d0 0x7f9584198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 --2- 172.21.15.107:0/3971521851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95841019c0 0x7f9584198140 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.551+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 >> 172.21.15.107:0/3971521851 conn(0x7f95840fd410 msgr2=0x7f95840ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.552+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 shutdown_connections 2024-08-20T06:35:13.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:13.552+0000 7f958b66e640 1 -- 172.21.15.107:0/3971521851 wait complete. 2024-08-20T06:35:13.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:35:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:35:13.535+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:35:13.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:35:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:35:13.535+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:35:13.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:35:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:35:13.535+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:35:13.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:35:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:35:13.535+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:35:13.850 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:35:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:35:13.535+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:35:14.209 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-20T06:35:14.462 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 -- 172.21.15.107:0/650484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c1073a0 msgr2=0x7fa57c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 --2- 172.21.15.107:0/650484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c1073a0 0x7fa57c1077e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 -- 172.21.15.107:0/650484579 shutdown_connections 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 --2- 172.21.15.107:0/650484579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c107d20 0x7fa57c10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 --2- 172.21.15.107:0/650484579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c1073a0 0x7fa57c1077e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 --2- 172.21.15.107:0/650484579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c101990 0x7fa57c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 -- 172.21.15.107:0/650484579 >> 172.21.15.107:0/650484579 conn(0x7fa57c0fd400 msgr2=0x7fa57c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 -- 172.21.15.107:0/650484579 shutdown_connections 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.947+0000 7fa5813ee640 1 -- 172.21.15.107:0/650484579 wait complete. 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 Processor -- start 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 -- start start 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c101990 0x7fa57c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:15.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 0x7fa57c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c111890 con 0x7fa57c107d20 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c111710 con 0x7fa57c101990 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa5813ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c111a10 con 0x7fa57c1073a0 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa57b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa57b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59996/0 (socket says 172.21.15.107:59996) 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa57b7fe640 1 -- 172.21.15.107:0/2834079351 learned_addr learned my addr 172.21.15.107:0/2834079351 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.948+0000 7fa573fff640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 0x7fa57c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 -- 172.21.15.107:0/2834079351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 msgr2=0x7fa57c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 0x7fa57c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57affd640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c101990 0x7fa57c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:15.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 -- 172.21.15.107:0/2834079351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c101990 msgr2=0x7fa57c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c101990 0x7fa57c198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 -- 172.21.15.107:0/2834079351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c19efd0 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa573fff640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 0x7fa57c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa57b7fe640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa57400eaa0 tx=0x7fa57400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57400cdb0 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa574005020 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c19f2c0 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740116d0 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.949+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c1a5980 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.950+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c101d70 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.952+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) ==== 151060+0+0 (secure 0 0 0) 0x7fa5740051d0 con 0x7fa57c107d20 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.953+0000 7fa578ff9640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 0x7fa5680b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.953+0000 7fa57affd640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 0x7fa5680b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:15.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.953+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fa5740e4110 con 0x7fa57c107d20 2024-08-20T06:35:15.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.953+0000 7fa57affd640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 0x7fa5680b6090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00a810 tx=0x7fa56c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:15.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:14.956+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa574016030 con 0x7fa57c107d20 2024-08-20T06:35:15.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.161+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa57c0fffd0 con 0x7fa5680b3bd0 2024-08-20T06:35:15.162 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:14 smithi107 ceph-mon[18495]: pgmap v277: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-08-20T06:35:15.163 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:14 smithi107 ceph-mon[18495]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:15.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.186+0000 7fa578ff9640 1 -- 172.21.15.107:0/2834079351 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa57c0fffd0 con 0x7fa5680b3bd0 2024-08-20T06:35:15.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 msgr2=0x7fa5680b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:15.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 0x7fa5680b6090 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa56c00a810 tx=0x7fa56c008040 comp rx=0 tx=0).stop 2024-08-20T06:35:15.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 msgr2=0x7fa57c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:15.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa57400eaa0 tx=0x7fa57400ef60 comp rx=0 tx=0).stop 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 shutdown_connections 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa57c107d20 0x7fa57c19e800 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa5680b3bd0 0x7fa5680b6090 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c1073a0 0x7fa57c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 --2- 172.21.15.107:0/2834079351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa57c101990 0x7fa57c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 >> 172.21.15.107:0/2834079351 conn(0x7fa57c0fd400 msgr2=0x7fa57c06ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 shutdown_connections 2024-08-20T06:35:15.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:15.188+0000 7fa5813ee640 1 -- 172.21.15.107:0/2834079351 wait complete. 2024-08-20T06:35:15.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:14 smithi183 ceph-mon[22204]: pgmap v277: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-08-20T06:35:15.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:14 smithi183 ceph-mon[22204]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:14 smithi193 ceph-mon[21944]: pgmap v277: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-08-20T06:35:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:14 smithi193 ceph-mon[21944]: from='client.14853 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:15.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-20T06:35:16.132 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:16.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:15 smithi107 ceph-mon[18495]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:16.296 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:15 smithi193 ceph-mon[21944]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:16.304 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:15 smithi183 ceph-mon[22204]: from='client.14865 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:16.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 -- 172.21.15.107:0/2036394963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 msgr2=0x7f9170107050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:16.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 --2- 172.21.15.107:0/2036394963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f9170107050 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9168009ca0 tx=0x7f916802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 -- 172.21.15.107:0/2036394963 shutdown_connections 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 --2- 172.21.15.107:0/2036394963 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f9170107050 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 --2- 172.21.15.107:0/2036394963 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 0x7f91701065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 --2- 172.21.15.107:0/2036394963 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 0x7f9170101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 -- 172.21.15.107:0/2036394963 >> 172.21.15.107:0/2036394963 conn(0x7f91700fd410 msgr2=0x7f91700ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 -- 172.21.15.107:0/2036394963 shutdown_connections 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.611+0000 7f9177488640 1 -- 172.21.15.107:0/2036394963 wait complete. 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 Processor -- start 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 -- start start 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 0x7f917019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 0x7f917019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:16.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9170118f90 con 0x7f9170106c00 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9170118e10 con 0x7f91701020e0 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9177488640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9170119110 con 0x7f91701017c0 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55702/0 (socket says 172.21.15.107:55702) 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 -- 172.21.15.107:0/3484822918 learned_addr learned my addr 172.21.15.107:0/3484822918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175224640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 0x7f917019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9174a23640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 0x7f917019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 -- 172.21.15.107:0/3484822918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 msgr2=0x7f917019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 0x7f917019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 -- 172.21.15.107:0/3484822918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 msgr2=0x7f917019fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 0x7f917019fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.612+0000 7f9175a25640 1 -- 172.21.15.107:0/3484822918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91701a5860 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f9175a25640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9168005e30 tx=0x7f91680093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168032d10 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9168032eb0 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168030b20 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91701a5af0 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.613+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91701ac210 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.614+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91701065b0 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.615+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f9168030ce0 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.616+0000 7f91667fc640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 0x7f915c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.616+0000 7f9175224640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 0x7f915c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.616+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f9168104de0 con 0x7f9170106c00 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.616+0000 7f9175224640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 0x7f915c0b6120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f916c009bd0 tx=0x7f916c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:16.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.620+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f91680cc590 con 0x7f9170106c00 2024-08-20T06:35:16.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.821+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f91700008d0 con 0x7f915c0b3c60 2024-08-20T06:35:16.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.825+0000 7f91667fc640 1 -- 172.21.15.107:0/3484822918 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f91700008d0 con 0x7f915c0b3c60 2024-08-20T06:35:16.826 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_0/6bb58bf9-4f8e-4e40-be28-10b8c824fb5d 2024-08-20T06:35:16.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.827+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 msgr2=0x7f915c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.827+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 0x7f915c0b6120 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f916c009bd0 tx=0x7f916c005f90 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.827+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 msgr2=0x7f91701a5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.827+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9168005e30 tx=0x7f91680093d0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 shutdown_connections 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9170106c00 0x7f91701a5090 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f91701020e0 0x7f917019fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f915c0b3c60 0x7f915c0b6120 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 --2- 172.21.15.107:0/3484822918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91701017c0 0x7f917019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 >> 172.21.15.107:0/3484822918 conn(0x7f91700fd410 msgr2=0x7f9170104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 shutdown_connections 2024-08-20T06:35:16.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:16.828+0000 7f9177488640 1 -- 172.21.15.107:0/3484822918 wait complete. 2024-08-20T06:35:17.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:16 smithi107 ceph-mon[18495]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-20T06:35:17.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:16 smithi107 ceph-mon[18495]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-20T06:35:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:16 smithi193 ceph-mon[21944]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-20T06:35:17.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:16 smithi193 ceph-mon[21944]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-20T06:35:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:16 smithi183 ceph-mon[22204]: pgmap v278: 97 pgs: 97 active+clean; 582 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-08-20T06:35:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:16 smithi183 ceph-mon[22204]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-08-20T06:35:17.543 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-20T06:35:17.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:18.171 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:17 smithi107 ceph-mon[18495]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:18.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 -- 172.21.15.107:0/2877657225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87501039a0 msgr2=0x7f8750103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 --2- 172.21.15.107:0/2877657225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87501039a0 0x7f8750103d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f874c009900 tx=0x7f874c02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 -- 172.21.15.107:0/2877657225 shutdown_connections 2024-08-20T06:35:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 --2- 172.21.15.107:0/2877657225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f875010ce00 0x7f875010f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 --2- 172.21.15.107:0/2877657225 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f875010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 --2- 172.21.15.107:0/2877657225 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f87501039a0 0x7f8750103d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 -- 172.21.15.107:0/2877657225 >> 172.21.15.107:0/2877657225 conn(0x7f87500fd400 msgr2=0x7f87500ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 -- 172.21.15.107:0/2877657225 shutdown_connections 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.255+0000 7f8757dee640 1 -- 172.21.15.107:0/2877657225 wait complete. 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 Processor -- start 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 -- start start 2024-08-20T06:35:18.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87501039a0 0x7f8750198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:18.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f87501993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:18.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:18.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8750112920 con 0x7f875010ce00 2024-08-20T06:35:18.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f87501127a0 con 0x7f87501042c0 2024-08-20T06:35:18.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8757dee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8750112aa0 con 0x7f87501039a0 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f875638b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f875638b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55730/0 (socket says 172.21.15.107:55730) 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f875638b640 1 -- 172.21.15.107:0/706329691 learned_addr learned my addr 172.21.15.107:0/706329691 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.256+0000 7f8755b8a640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87501039a0 0x7f8750198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8755389640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f87501993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:18.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 -- 172.21.15.107:0/706329691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87501039a0 msgr2=0x7f8750198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87501039a0 0x7f8750198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 -- 172.21.15.107:0/706329691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 msgr2=0x7f87501993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f87501993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 -- 172.21.15.107:0/706329691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875019ef40 con 0x7f875010ce00 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8755389640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f87501993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:18.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f875638b640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f874800eaa0 tx=0x7f874800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874800cdb0 con 0x7f875010ce00 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8748005020 con 0x7f875010ce00 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875019f230 con 0x7f875010ce00 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87480116d0 con 0x7f875010ce00 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.257+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87501a58f0 con 0x7f875010ce00 2024-08-20T06:35:18.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.258+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8750103d80 con 0x7f875010ce00 2024-08-20T06:35:18.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.259+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f87480051d0 con 0x7f875010ce00 2024-08-20T06:35:18.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.260+0000 7f8746ffd640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 0x7f873c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:18.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.260+0000 7f8755b8a640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 0x7f873c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:18.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.260+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f87480e3d90 con 0x7f875010ce00 2024-08-20T06:35:18.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.261+0000 7f8755b8a640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 0x7f873c0b6120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f874c0057c0 tx=0x7f874c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:18.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.264+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8748016030 con 0x7f875010ce00 2024-08-20T06:35:18.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:17 smithi193 ceph-mon[21944]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:18.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.466+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f87500632c0 con 0x7f873c0b3c60 2024-08-20T06:35:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:17 smithi183 ceph-mon[22204]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:18.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.488+0000 7f8746ffd640 1 -- 172.21.15.107:0/706329691 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f87500632c0 con 0x7f873c0b3c60 2024-08-20T06:35:18.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 msgr2=0x7f873c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:18.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 0x7f873c0b6120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f874c0057c0 tx=0x7f874c0058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 msgr2=0x7f875019e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:18.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f874800eaa0 tx=0x7f874800ef60 comp rx=0 tx=0).stop 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 shutdown_connections 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f875010ce00 0x7f875019e770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f87501042c0 0x7f87501993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f873c0b3c60 0x7f873c0b6120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 --2- 172.21.15.107:0/706329691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87501039a0 0x7f8750198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 >> 172.21.15.107:0/706329691 conn(0x7f87500fd400 msgr2=0x7f87500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 shutdown_connections 2024-08-20T06:35:18.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:18.491+0000 7f8757dee640 1 -- 172.21.15.107:0/706329691 wait complete. 2024-08-20T06:35:19.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:18 smithi107 ceph-mon[18495]: pgmap v279: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T06:35:19.146 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-20T06:35:19.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:18 smithi193 ceph-mon[21944]: pgmap v279: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T06:35:19.391 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:18 smithi183 ceph-mon[22204]: pgmap v279: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 -- 172.21.15.107:0/3373427389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8158104cb0 msgr2=0x7f8158105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 --2- 172.21.15.107:0/3373427389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8158104cb0 0x7f8158105090 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8154009900 tx=0x7f815402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 -- 172.21.15.107:0/3373427389 shutdown_connections 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 --2- 172.21.15.107:0/3373427389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815810b040 0x7f8158111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 --2- 172.21.15.107:0/3373427389 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 0x7f815810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:19.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 --2- 172.21.15.107:0/3373427389 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8158104cb0 0x7f8158105090 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 -- 172.21.15.107:0/3373427389 >> 172.21.15.107:0/3373427389 conn(0x7f8158100720 msgr2=0x7f8158102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 -- 172.21.15.107:0/3373427389 shutdown_connections 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.880+0000 7f81602af640 1 -- 172.21.15.107:0/3373427389 wait complete. 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 Processor -- start 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 -- start start 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8158104cb0 0x7f815819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 0x7f815819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8158114bb0 con 0x7f815810b040 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8158114a30 con 0x7f815810a6c0 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f81602af640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8158114d30 con 0x7f8158104cb0 2024-08-20T06:35:19.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.881+0000 7f815e84c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55748/0 (socket says 172.21.15.107:55748) 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 -- 172.21.15.107:0/1788456293 learned_addr learned my addr 172.21.15.107:0/1788456293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815d84a640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 0x7f815819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e04b640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8158104cb0 0x7f815819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 -- 172.21.15.107:0/1788456293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8158104cb0 msgr2=0x7f815819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8158104cb0 0x7f815819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 -- 172.21.15.107:0/1788456293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 msgr2=0x7f815819b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 0x7f815819b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:19.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 -- 172.21.15.107:0/1788456293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81581a1190 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f815e84c640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f815000b970 tx=0x7f815000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150019070 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8150014440 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81581a1480 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81500075d0 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.882+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81581a7b40 con 0x7f815810b040 2024-08-20T06:35:19.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.883+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8158105090 con 0x7f815810b040 2024-08-20T06:35:19.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.885+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f815001c070 con 0x7f815810b040 2024-08-20T06:35:19.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.886+0000 7f814f7fe640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 0x7f81440b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:19.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.886+0000 7f815e04b640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 0x7f81440b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:19.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.886+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f81500e35e0 con 0x7f815810b040 2024-08-20T06:35:19.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.886+0000 7f815e04b640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 0x7f81440b6120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81540057c0 tx=0x7f81540058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:19.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:19.889+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f81500aa640 con 0x7f815810b040 2024-08-20T06:35:20.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.093+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f81580632c0 con 0x7f81440b3c60 2024-08-20T06:35:20.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:19 smithi107 ceph-mon[18495]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:20.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.098+0000 7f814f7fe640 1 -- 172.21.15.107:0/1788456293 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f81580632c0 con 0x7f81440b3c60 2024-08-20T06:35:20.099 INFO:teuthology.orchestra.run.smithi107.stdout:/volumes/qa/sv_1/3c973a02-9980-4e54-bed5-557aa7d1b5fb 2024-08-20T06:35:20.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 msgr2=0x7f81440b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 0x7f81440b6120 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f81540057c0 tx=0x7f81540058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 msgr2=0x7f81581a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f815000b970 tx=0x7f815000be30 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 shutdown_connections 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815810b040 0x7f81581a09c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815810a6c0 0x7f815819b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81440b3c60 0x7f81440b6120 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 --2- 172.21.15.107:0/1788456293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8158104cb0 0x7f815819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.100+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 >> 172.21.15.107:0/1788456293 conn(0x7f8158100720 msgr2=0x7f8158102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.101+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 shutdown_connections 2024-08-20T06:35:20.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:20.101+0000 7f81602af640 1 -- 172.21.15.107:0/1788456293 wait complete. 2024-08-20T06:35:20.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:19 smithi193 ceph-mon[21944]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:19 smithi183 ceph-mon[22204]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:20.788 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:35:21.035 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:21.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:21 smithi107 ceph-mon[18495]: pgmap v280: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-20T06:35:21.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:21 smithi107 ceph-mon[18495]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:21 smithi193 ceph-mon[21944]: pgmap v280: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-20T06:35:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:21 smithi193 ceph-mon[21944]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:21 smithi183 ceph-mon[22204]: pgmap v280: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.4 KiB/s wr, 13 op/s 2024-08-20T06:35:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:21 smithi183 ceph-mon[22204]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- 172.21.15.107:0/3174037684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10108e60 msgr2=0x7f4f1010f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3174037684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10108e60 0x7f4f1010f660 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f08009ca0 tx=0x7f4f0802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- 172.21.15.107:0/3174037684 shutdown_connections 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3174037684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f10108e60 0x7f4f1010f660 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3174037684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f101084e0 0x7f4f10108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3174037684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10102ad0 0x7f4f10102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- 172.21.15.107:0/3174037684 >> 172.21.15.107:0/3174037684 conn(0x7f4f100fe540 msgr2=0x7f4f10100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- 172.21.15.107:0/3174037684 shutdown_connections 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- 172.21.15.107:0/3174037684 wait complete. 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 Processor -- start 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.474+0000 7f4f16b16640 1 -- start start 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10102ad0 0x7f4f10198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 0x7f4f1019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10112a20 con 0x7f4f101084e0 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f101128a0 con 0x7f4f10108e60 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f10112ba0 con 0x7f4f10102ad0 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55776/0 (socket says 172.21.15.107:55776) 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 -- 172.21.15.107:0/3411835715 learned_addr learned my addr 172.21.15.107:0/3411835715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f148b2640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10102ad0 0x7f4f10198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f150b3640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 0x7f4f1019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 -- 172.21.15.107:0/3411835715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10102ad0 msgr2=0x7f4f10198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10102ad0 0x7f4f10198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 -- 172.21.15.107:0/3411835715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 msgr2=0x7f4f1019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 0x7f4f1019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 -- 172.21.15.107:0/3411835715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1019efa0 con 0x7f4f101084e0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f150b3640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 0x7f4f1019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0ffff640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d770 tx=0x7f4efc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc017070 con 0x7f4f101084e0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4efc0054b0 con 0x7f4f101084e0 2024-08-20T06:35:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1019f290 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.475+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc004c60 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.476+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f101a5950 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.477+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f10102eb0 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.478+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f4efc004e00 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.479+0000 7f4f0dffb640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 0x7f4f000b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.479+0000 7f4f148b2640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 0x7f4f000b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.479+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f4efc0e3730 con 0x7f4f101084e0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.479+0000 7f4f148b2640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 0x7f4f000b6120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009890 tx=0x7f4f04008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.482+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4efc0aadd0 con 0x7f4f101084e0 2024-08-20T06:35:21.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.710+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1019a020 con 0x7f4f101084e0 2024-08-20T06:35:21.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.710+0000 7f4f0dffb640 1 -- 172.21.15.107:0/3411835715 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f4efc0aa6d0 con 0x7f4f101084e0 2024-08-20T06:35:21.712 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:21.712 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:35:21.713 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:35:21.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.712+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 msgr2=0x7f4f000b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:21.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.712+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 0x7f4f000b6120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009890 tx=0x7f4f04008040 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.712+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 msgr2=0x7f4f10199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.712+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00d770 tx=0x7f4efc00dc30 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 shutdown_connections 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f10108e60 0x7f4f1019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f101084e0 0x7f4f10199430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f000b3c60 0x7f4f000b6120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 --2- 172.21.15.107:0/3411835715 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f10102ad0 0x7f4f10198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 >> 172.21.15.107:0/3411835715 conn(0x7f4f100fe540 msgr2=0x7f4f10100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 shutdown_connections 2024-08-20T06:35:21.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:21.713+0000 7f4f16b16640 1 -- 172.21.15.107:0/3411835715 wait complete. 2024-08-20T06:35:22.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:22 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3411835715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:35:22.343 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:22 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3411835715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:35:22.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:35:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:22 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3411835715' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:35:22.617 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:23.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.066+0000 7f690bd49640 1 -- 172.21.15.107:0/2012364017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69041039a0 msgr2=0x7f6904103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:23.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.066+0000 7f690bd49640 1 --2- 172.21.15.107:0/2012364017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69041039a0 0x7f6904103d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68f4009ca0 tx=0x7f68f402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- 172.21.15.107:0/2012364017 shutdown_connections 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- 172.21.15.107:0/2012364017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690410ce00 0x7f690410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- 172.21.15.107:0/2012364017 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f690410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- 172.21.15.107:0/2012364017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f69041039a0 0x7f6904103d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- 172.21.15.107:0/2012364017 >> 172.21.15.107:0/2012364017 conn(0x7f69040fd400 msgr2=0x7f69040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- 172.21.15.107:0/2012364017 shutdown_connections 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- 172.21.15.107:0/2012364017 wait complete. 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 Processor -- start 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- start start 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69041039a0 0x7f6904198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f6904199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6904112a50 con 0x7f690410ce00 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f69041128d0 con 0x7f69041042c0 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.067+0000 7f690bd49640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6904112bd0 con 0x7f69041039a0 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55800/0 (socket says 172.21.15.107:55800) 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 -- 172.21.15.107:0/3995397219 learned_addr learned my addr 172.21.15.107:0/3995397219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f6909ae5640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69041039a0 0x7f6904198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f69092e4640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f6904199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 -- 172.21.15.107:0/3995397219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69041039a0 msgr2=0x7f6904198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69041039a0 0x7f6904198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 -- 172.21.15.107:0/3995397219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 msgr2=0x7f6904199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:23.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f6904199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 -- 172.21.15.107:0/3995397219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690419f070 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f69092e4640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f6904199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690a2e6640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009fd0 tx=0x7f68fc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc011830 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f68fc016070 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690419f360 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc010d20 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.068+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69041a5aa0 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.069+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6904103d80 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.070+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f68fc01b460 con 0x7f690410ce00 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.071+0000 7f6902ffd640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 0x7f68ec0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.072+0000 7f6909ae5640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 0x7f68ec0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.072+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f68fc0e31f0 con 0x7f690410ce00 2024-08-20T06:35:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.072+0000 7f6909ae5640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 0x7f68ec0b6120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68f4031040 tx=0x7f68f4000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.076+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f68fc0aa9a0 con 0x7f690410ce00 2024-08-20T06:35:23.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.252+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69041a5ed0 con 0x7f690410ce00 2024-08-20T06:35:23.253 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:23 smithi107 ceph-mon[18495]: pgmap v281: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-08-20T06:35:23.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.254+0000 7f6902ffd640 1 -- 172.21.15.107:0/3995397219 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+22371 (secure 0 0 0) 0x7f68fc0aa2a0 con 0x7f690410ce00 2024-08-20T06:35:23.255 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:23.255 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":71,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:35:00.478609+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 msgr2=0x7f68ec0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 0x7f68ec0b6120 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f68f4031040 tx=0x7f68f4000f40 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 msgr2=0x7f690419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009fd0 tx=0x7f68fc00b8e0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 shutdown_connections 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f690410ce00 0x7f690419e8a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f69041042c0 0x7f6904199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f68ec0b3c60 0x7f68ec0b6120 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 --2- 172.21.15.107:0/3995397219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69041039a0 0x7f6904198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:23.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 >> 172.21.15.107:0/3995397219 conn(0x7f69040fd400 msgr2=0x7f69040fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:23.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 shutdown_connections 2024-08-20T06:35:23.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:23.256+0000 7f690bd49640 1 -- 172.21.15.107:0/3995397219 wait complete. 2024-08-20T06:35:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:23 smithi193 ceph-mon[21944]: pgmap v281: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-08-20T06:35:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:23 smithi183 ceph-mon[22204]: pgmap v281: 97 pgs: 97 active+clean; 606 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.1 KiB/s wr, 13 op/s 2024-08-20T06:35:23.951 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T06:35:23.951 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:35:24.200 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:24.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:24 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3995397219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:24.320 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:24 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3995397219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:24 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3995397219' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:24.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 -- 172.21.15.107:0/1594843623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 msgr2=0x7f2d78102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:24.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 --2- 172.21.15.107:0/1594843623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78102480 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009900 tx=0x7f2d7402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:24.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 -- 172.21.15.107:0/1594843623 shutdown_connections 2024-08-20T06:35:24.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 --2- 172.21.15.107:0/1594843623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 0x7f2d7810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 --2- 172.21.15.107:0/1594843623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 0x7f2d781045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 --2- 172.21.15.107:0/1594843623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78102480 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 -- 172.21.15.107:0/1594843623 >> 172.21.15.107:0/1594843623 conn(0x7f2d78077e30 msgr2=0x7f2d780ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 -- 172.21.15.107:0/1594843623 shutdown_connections 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.710+0000 7f2d7f342640 1 -- 172.21.15.107:0/1594843623 wait complete. 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 Processor -- start 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 -- start start 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 0x7f2d78199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 0x7f2d7819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:24.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2d78111870 con 0x7f2d781020a0 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781116f0 con 0x7f2d78104b00 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7f342640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2d781119f0 con 0x7f2d78104180 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55814/0 (socket says 172.21.15.107:55814) 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 -- 172.21.15.107:0/3641508296 learned_addr learned my addr 172.21.15.107:0/3641508296 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7c8dd640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 0x7f2d78199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d8df640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 0x7f2d7819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 -- 172.21.15.107:0/3641508296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 msgr2=0x7f2d78199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 0x7f2d78199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 -- 172.21.15.107:0/3641508296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 msgr2=0x7f2d7819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 0x7f2d7819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.711+0000 7f2d7d0de640 1 -- 172.21.15.107:0/3641508296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d7819ef80 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d7d0de640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009230 tx=0x7f2d74004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7403f030 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2d74003a80 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d7819f210 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74035070 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.712+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d781a5930 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.713+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d78102480 con 0x7f2d781020a0 2024-08-20T06:35:24.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.714+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2d74042030 con 0x7f2d781020a0 2024-08-20T06:35:24.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.715+0000 7f2d6e7fc640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 0x7f2d640b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:24.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.715+0000 7f2d7c8dd640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 0x7f2d640b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:24.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.715+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f2d74104620 con 0x7f2d781020a0 2024-08-20T06:35:24.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.715+0000 7f2d7c8dd640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 0x7f2d640b61a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d781054b0 tx=0x7f2d6800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:24.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.718+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2d740cb670 con 0x7f2d781020a0 2024-08-20T06:35:24.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.888+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2d780ff3f0 con 0x7f2d640b3ce0 2024-08-20T06:35:24.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.893+0000 7f2d6e7fc640 1 -- 172.21.15.107:0/3641508296 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258693 (secure 0 0 0) 0x7f2d780ff3f0 con 0x7f2d640b3ce0 2024-08-20T06:35:24.897 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:24.898 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:35:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 msgr2=0x7f2d640b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 0x7f2d640b61a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f2d781054b0 tx=0x7f2d6800b390 comp rx=0 tx=0).stop 2024-08-20T06:35:24.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 msgr2=0x7f2d78198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d74009230 tx=0x7f2d74004270 comp rx=0 tx=0).stop 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 shutdown_connections 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2d78104b00 0x7f2d7819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2d640b3ce0 0x7f2d640b61a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d78104180 0x7f2d78199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 --2- 172.21.15.107:0/3641508296 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2d781020a0 0x7f2d78198ed0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 >> 172.21.15.107:0/3641508296 conn(0x7f2d78077e30 msgr2=0x7f2d78107b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 shutdown_connections 2024-08-20T06:35:24.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:24.896+0000 7f2d7f342640 1 -- 172.21.15.107:0/3641508296 wait complete. 2024-08-20T06:35:25.269 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:25 smithi107 ceph-mon[18495]: pgmap v282: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-20T06:35:25.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:25 smithi193 ceph-mon[21944]: pgmap v282: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-20T06:35:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:25 smithi183 ceph-mon[22204]: pgmap v282: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 5.4 KiB/s wr, 12 op/s 2024-08-20T06:35:25.598 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-08-20T06:35:23.310440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646310,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":177,"num_read_kb":174,"num_write":274,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364460,"kb_used_data":11052,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577364,"statfs":{"total":1151940427776,"available":1151567220736,"internally_reserved":0,"allocated":11317248,"data_stored":6331852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51612,"num_objects":16,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":90,"num_read_kb":100,"num_write":56,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020205"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926320+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:35:00.926320+0000","last_peered":"2024-08-20T06:35:00.926320+0000","last_clean":"2024-08-20T06:35:00.926320+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:35:00.926320+0000","last_undegraded":"2024-08-20T06:35:00.926320+0000","last_fullsized":"2024-08-20T06:35:00.926320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927883+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:35:00.927883+0000","last_peered":"2024-08-20T06:35:00.927883+0000","last_clean":"2024-08-20T06:35:00.927883+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:35:00.927883+0000","last_undegraded":"2024-08-20T06:35:00.927883+0000","last_fullsized":"2024-08-20T06:35:00.927883+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927658+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:35:00.927658+0000","last_peered":"2024-08-20T06:35:00.927658+0000","last_clean":"2024-08-20T06:35:00.927658+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:35:00.927658+0000","last_undegraded":"2024-08-20T06:35:00.927658+0000","last_fullsized":"2024-08-20T06:35:00.927658+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918161+0000","last_change":"2024-08-20T06:34:59.933963+0000","last_active":"2024-08-20T06:35:00.918161+0000","last_peered":"2024-08-20T06:35:00.918161+0000","last_clean":"2024-08-20T06:35:00.918161+0000","last_became_active":"2024-08-20T06:34:59.933682+0000","last_became_peered":"2024-08-20T06:34:59.933682+0000","last_unstale":"2024-08-20T06:35:00.918161+0000","last_undegraded":"2024-08-20T06:35:00.918161+0000","last_fullsized":"2024-08-20T06:35:00.918161+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:07.550290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924480+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:35:00.924480+0000","last_peered":"2024-08-20T06:35:00.924480+0000","last_clean":"2024-08-20T06:35:00.924480+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:35:00.924480+0000","last_undegraded":"2024-08-20T06:35:00.924480+0000","last_fullsized":"2024-08-20T06:35:00.924480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924408+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:35:00.924408+0000","last_peered":"2024-08-20T06:35:00.924408+0000","last_clean":"2024-08-20T06:35:00.924408+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:35:00.924408+0000","last_undegraded":"2024-08-20T06:35:00.924408+0000","last_fullsized":"2024-08-20T06:35:00.924408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925543+0000","last_change":"2024-08-20T06:34:59.933789+0000","last_active":"2024-08-20T06:35:00.925543+0000","last_peered":"2024-08-20T06:35:00.925543+0000","last_clean":"2024-08-20T06:35:00.925543+0000","last_became_active":"2024-08-20T06:34:59.933461+0000","last_became_peered":"2024-08-20T06:34:59.933461+0000","last_unstale":"2024-08-20T06:35:00.925543+0000","last_undegraded":"2024-08-20T06:35:00.925543+0000","last_fullsized":"2024-08-20T06:35:00.925543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:31:37.924168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924924+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:35:00.924924+0000","last_peered":"2024-08-20T06:35:00.924924+0000","last_clean":"2024-08-20T06:35:00.924924+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:35:00.924924+0000","last_undegraded":"2024-08-20T06:35:00.924924+0000","last_fullsized":"2024-08-20T06:35:00.924924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928568+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:35:00.928568+0000","last_peered":"2024-08-20T06:35:00.928568+0000","last_clean":"2024-08-20T06:35:00.928568+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:35:00.928568+0000","last_undegraded":"2024-08-20T06:35:00.928568+0000","last_fullsized":"2024-08-20T06:35:00.928568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925563+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:35:00.925563+0000","last_peered":"2024-08-20T06:35:00.925563+0000","last_clean":"2024-08-20T06:35:00.925563+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:35:00.925563+0000","last_undegraded":"2024-08-20T06:35:00.925563+0000","last_fullsized":"2024-08-20T06:35:00.925563+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926964+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:35:00.926964+0000","last_peered":"2024-08-20T06:35:00.926964+0000","last_clean":"2024-08-20T06:35:00.926964+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:35:00.926964+0000","last_undegraded":"2024-08-20T06:35:00.926964+0000","last_fullsized":"2024-08-20T06:35:00.926964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918213+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:35:00.918213+0000","last_peered":"2024-08-20T06:35:00.918213+0000","last_clean":"2024-08-20T06:35:00.918213+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:35:00.918213+0000","last_undegraded":"2024-08-20T06:35:00.918213+0000","last_fullsized":"2024-08-20T06:35:00.918213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924444+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:35:00.924444+0000","last_peered":"2024-08-20T06:35:00.924444+0000","last_clean":"2024-08-20T06:35:00.924444+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:35:00.924444+0000","last_undegraded":"2024-08-20T06:35:00.924444+0000","last_fullsized":"2024-08-20T06:35:00.924444+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927989+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:35:00.927989+0000","last_peered":"2024-08-20T06:35:00.927989+0000","last_clean":"2024-08-20T06:35:00.927989+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:35:00.927989+0000","last_undegraded":"2024-08-20T06:35:00.927989+0000","last_fullsized":"2024-08-20T06:35:00.927989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927600+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:35:00.927600+0000","last_peered":"2024-08-20T06:35:00.927600+0000","last_clean":"2024-08-20T06:35:00.927600+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:35:00.927600+0000","last_undegraded":"2024-08-20T06:35:00.927600+0000","last_fullsized":"2024-08-20T06:35:00.927600+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928257+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:35:00.928257+0000","last_peered":"2024-08-20T06:35:00.928257+0000","last_clean":"2024-08-20T06:35:00.928257+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:35:00.928257+0000","last_undegraded":"2024-08-20T06:35:00.928257+0000","last_fullsized":"2024-08-20T06:35:00.928257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926363+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:35:00.926363+0000","last_peered":"2024-08-20T06:35:00.926363+0000","last_clean":"2024-08-20T06:35:00.926363+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:35:00.926363+0000","last_undegraded":"2024-08-20T06:35:00.926363+0000","last_fullsized":"2024-08-20T06:35:00.926363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917047+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:35:00.917047+0000","last_peered":"2024-08-20T06:35:00.917047+0000","last_clean":"2024-08-20T06:35:00.917047+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:35:00.917047+0000","last_undegraded":"2024-08-20T06:35:00.917047+0000","last_fullsized":"2024-08-20T06:35:00.917047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917498+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:35:00.917498+0000","last_peered":"2024-08-20T06:35:00.917498+0000","last_clean":"2024-08-20T06:35:00.917498+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:35:00.917498+0000","last_undegraded":"2024-08-20T06:35:00.917498+0000","last_fullsized":"2024-08-20T06:35:00.917498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924758+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:35:00.924758+0000","last_peered":"2024-08-20T06:35:00.924758+0000","last_clean":"2024-08-20T06:35:00.924758+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:35:00.924758+0000","last_undegraded":"2024-08-20T06:35:00.924758+0000","last_fullsized":"2024-08-20T06:35:00.924758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924606+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:35:00.924606+0000","last_peered":"2024-08-20T06:35:00.924606+0000","last_clean":"2024-08-20T06:35:00.924606+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:35:00.924606+0000","last_undegraded":"2024-08-20T06:35:00.924606+0000","last_fullsized":"2024-08-20T06:35:00.924606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928457+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:35:00.928457+0000","last_peered":"2024-08-20T06:35:00.928457+0000","last_clean":"2024-08-20T06:35:00.928457+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:35:00.928457+0000","last_undegraded":"2024-08-20T06:35:00.928457+0000","last_fullsized":"2024-08-20T06:35:00.928457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924874+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:35:00.924874+0000","last_peered":"2024-08-20T06:35:00.924874+0000","last_clean":"2024-08-20T06:35:00.924874+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:35:00.924874+0000","last_undegraded":"2024-08-20T06:35:00.924874+0000","last_fullsized":"2024-08-20T06:35:00.924874+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927847+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:35:00.927847+0000","last_peered":"2024-08-20T06:35:00.927847+0000","last_clean":"2024-08-20T06:35:00.927847+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:35:00.927847+0000","last_undegraded":"2024-08-20T06:35:00.927847+0000","last_fullsized":"2024-08-20T06:35:00.927847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925294+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:35:00.925294+0000","last_peered":"2024-08-20T06:35:00.925294+0000","last_clean":"2024-08-20T06:35:00.925294+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:35:00.925294+0000","last_undegraded":"2024-08-20T06:35:00.925294+0000","last_fullsized":"2024-08-20T06:35:00.925294+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917752+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:35:00.917752+0000","last_peered":"2024-08-20T06:35:00.917752+0000","last_clean":"2024-08-20T06:35:00.917752+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:35:00.917752+0000","last_undegraded":"2024-08-20T06:35:00.917752+0000","last_fullsized":"2024-08-20T06:35:00.917752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925219+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:35:00.925219+0000","last_peered":"2024-08-20T06:35:00.925219+0000","last_clean":"2024-08-20T06:35:00.925219+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:35:00.925219+0000","last_undegraded":"2024-08-20T06:35:00.925219+0000","last_fullsized":"2024-08-20T06:35:00.925219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925939+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:35:00.925939+0000","last_peered":"2024-08-20T06:35:00.925939+0000","last_clean":"2024-08-20T06:35:00.925939+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:35:00.925939+0000","last_undegraded":"2024-08-20T06:35:00.925939+0000","last_fullsized":"2024-08-20T06:35:00.925939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928092+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:35:00.928092+0000","last_peered":"2024-08-20T06:35:00.928092+0000","last_clean":"2024-08-20T06:35:00.928092+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:35:00.928092+0000","last_undegraded":"2024-08-20T06:35:00.928092+0000","last_fullsized":"2024-08-20T06:35:00.928092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917181+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:35:00.917181+0000","last_peered":"2024-08-20T06:35:00.917181+0000","last_clean":"2024-08-20T06:35:00.917181+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:35:00.917181+0000","last_undegraded":"2024-08-20T06:35:00.917181+0000","last_fullsized":"2024-08-20T06:35:00.917181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917419+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:35:00.917419+0000","last_peered":"2024-08-20T06:35:00.917419+0000","last_clean":"2024-08-20T06:35:00.917419+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:35:00.917419+0000","last_undegraded":"2024-08-20T06:35:00.917419+0000","last_fullsized":"2024-08-20T06:35:00.917419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924729+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:35:00.924729+0000","last_peered":"2024-08-20T06:35:00.924729+0000","last_clean":"2024-08-20T06:35:00.924729+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:35:00.924729+0000","last_undegraded":"2024-08-20T06:35:00.924729+0000","last_fullsized":"2024-08-20T06:35:00.924729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924280+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:35:00.924280+0000","last_peered":"2024-08-20T06:35:00.924280+0000","last_clean":"2024-08-20T06:35:00.924280+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:35:00.924280+0000","last_undegraded":"2024-08-20T06:35:00.924280+0000","last_fullsized":"2024-08-20T06:35:00.924280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":37,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:18.998842+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:35:18.998842+0000","last_peered":"2024-08-20T06:35:18.998842+0000","last_clean":"2024-08-20T06:35:18.998842+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:35:18.998842+0000","last_undegraded":"2024-08-20T06:35:18.998842+0000","last_fullsized":"2024-08-20T06:35:18.998842+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928023+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:35:00.928023+0000","last_peered":"2024-08-20T06:35:00.928023+0000","last_clean":"2024-08-20T06:35:00.928023+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:35:00.928023+0000","last_undegraded":"2024-08-20T06:35:00.928023+0000","last_fullsized":"2024-08-20T06:35:00.928023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973053+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:35:08.973053+0000","last_peered":"2024-08-20T06:35:08.973053+0000","last_clean":"2024-08-20T06:35:08.973053+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:35:08.973053+0000","last_undegraded":"2024-08-20T06:35:08.973053+0000","last_fullsized":"2024-08-20T06:35:08.973053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971914+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:35:08.971914+0000","last_peered":"2024-08-20T06:35:08.971914+0000","last_clean":"2024-08-20T06:35:08.971914+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:35:08.971914+0000","last_undegraded":"2024-08-20T06:35:08.971914+0000","last_fullsized":"2024-08-20T06:35:08.971914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925068+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:35:00.925068+0000","last_peered":"2024-08-20T06:35:00.925068+0000","last_clean":"2024-08-20T06:35:00.925068+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:35:00.925068+0000","last_undegraded":"2024-08-20T06:35:00.925068+0000","last_fullsized":"2024-08-20T06:35:00.925068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928244+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:35:00.928244+0000","last_peered":"2024-08-20T06:35:00.928244+0000","last_clean":"2024-08-20T06:35:00.928244+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:35:00.928244+0000","last_undegraded":"2024-08-20T06:35:00.928244+0000","last_fullsized":"2024-08-20T06:35:00.928244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916637+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:35:00.916637+0000","last_peered":"2024-08-20T06:35:00.916637+0000","last_clean":"2024-08-20T06:35:00.916637+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:35:00.916637+0000","last_undegraded":"2024-08-20T06:35:00.916637+0000","last_fullsized":"2024-08-20T06:35:00.916637+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:21.000990+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:35:21.000990+0000","last_peered":"2024-08-20T06:35:21.000990+0000","last_clean":"2024-08-20T06:35:21.000990+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:35:21.000990+0000","last_undegraded":"2024-08-20T06:35:21.000990+0000","last_fullsized":"2024-08-20T06:35:21.000990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928449+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:35:00.928449+0000","last_peered":"2024-08-20T06:35:00.928449+0000","last_clean":"2024-08-20T06:35:00.928449+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:35:00.928449+0000","last_undegraded":"2024-08-20T06:35:00.928449+0000","last_fullsized":"2024-08-20T06:35:00.928449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965368+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:35:07.965368+0000","last_peered":"2024-08-20T06:35:07.965368+0000","last_clean":"2024-08-20T06:35:07.965368+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:35:07.965368+0000","last_undegraded":"2024-08-20T06:35:07.965368+0000","last_fullsized":"2024-08-20T06:35:07.965368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928137+0000","last_change":"2024-08-20T06:34:59.930088+0000","last_active":"2024-08-20T06:35:00.928137+0000","last_peered":"2024-08-20T06:35:00.928137+0000","last_clean":"2024-08-20T06:35:00.928137+0000","last_became_active":"2024-08-20T06:34:59.929780+0000","last_became_peered":"2024-08-20T06:34:59.929780+0000","last_unstale":"2024-08-20T06:35:00.928137+0000","last_undegraded":"2024-08-20T06:35:00.928137+0000","last_fullsized":"2024-08-20T06:35:00.928137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:11.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964783+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:35:07.964783+0000","last_peered":"2024-08-20T06:35:07.964783+0000","last_clean":"2024-08-20T06:35:07.964783+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:35:07.964783+0000","last_undegraded":"2024-08-20T06:35:07.964783+0000","last_fullsized":"2024-08-20T06:35:07.964783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928799+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:35:00.928799+0000","last_peered":"2024-08-20T06:35:00.928799+0000","last_clean":"2024-08-20T06:35:00.928799+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:35:00.928799+0000","last_undegraded":"2024-08-20T06:35:00.928799+0000","last_fullsized":"2024-08-20T06:35:00.928799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'26","reported_seq":77,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:22.004860+0000","last_change":"2024-08-20T06:34:57.920527+0000","last_active":"2024-08-20T06:35:22.004860+0000","last_peered":"2024-08-20T06:35:22.004860+0000","last_clean":"2024-08-20T06:35:22.004860+0000","last_became_active":"2024-08-20T06:34:57.920116+0000","last_became_peered":"2024-08-20T06:34:57.920116+0000","last_unstale":"2024-08-20T06:35:22.004860+0000","last_undegraded":"2024-08-20T06:35:22.004860+0000","last_fullsized":"2024-08-20T06:35:22.004860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:46:06.121047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926259+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:35:00.926259+0000","last_peered":"2024-08-20T06:35:00.926259+0000","last_clean":"2024-08-20T06:35:00.926259+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:35:00.926259+0000","last_undegraded":"2024-08-20T06:35:00.926259+0000","last_fullsized":"2024-08-20T06:35:00.926259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972040+0000","last_change":"2024-08-20T06:34:57.919137+0000","last_active":"2024-08-20T06:35:08.972040+0000","last_peered":"2024-08-20T06:35:08.972040+0000","last_clean":"2024-08-20T06:35:08.972040+0000","last_became_active":"2024-08-20T06:34:57.918821+0000","last_became_peered":"2024-08-20T06:34:57.918821+0000","last_unstale":"2024-08-20T06:35:08.972040+0000","last_undegraded":"2024-08-20T06:35:08.972040+0000","last_fullsized":"2024-08-20T06:35:08.972040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:51:18.267609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.923922+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:35:00.923922+0000","last_peered":"2024-08-20T06:35:00.923922+0000","last_clean":"2024-08-20T06:35:00.923922+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:35:00.923922+0000","last_undegraded":"2024-08-20T06:35:00.923922+0000","last_fullsized":"2024-08-20T06:35:00.923922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964646+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:35:07.964646+0000","last_peered":"2024-08-20T06:35:07.964646+0000","last_clean":"2024-08-20T06:35:07.964646+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:35:07.964646+0000","last_undegraded":"2024-08-20T06:35:07.964646+0000","last_fullsized":"2024-08-20T06:35:07.964646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925098+0000","last_change":"2024-08-20T06:34:59.930132+0000","last_active":"2024-08-20T06:35:00.925098+0000","last_peered":"2024-08-20T06:35:00.925098+0000","last_clean":"2024-08-20T06:35:00.925098+0000","last_became_active":"2024-08-20T06:34:59.929820+0000","last_became_peered":"2024-08-20T06:34:59.929820+0000","last_unstale":"2024-08-20T06:35:00.925098+0000","last_undegraded":"2024-08-20T06:35:00.925098+0000","last_fullsized":"2024-08-20T06:35:00.925098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:30.947526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:22.002972+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:35:22.002972+0000","last_peered":"2024-08-20T06:35:22.002972+0000","last_clean":"2024-08-20T06:35:22.002972+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:35:22.002972+0000","last_undegraded":"2024-08-20T06:35:22.002972+0000","last_fullsized":"2024-08-20T06:35:22.002972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924673+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:35:00.924673+0000","last_peered":"2024-08-20T06:35:00.924673+0000","last_clean":"2024-08-20T06:35:00.924673+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:35:00.924673+0000","last_undegraded":"2024-08-20T06:35:00.924673+0000","last_fullsized":"2024-08-20T06:35:00.924673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973062+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:35:08.973062+0000","last_peered":"2024-08-20T06:35:08.973062+0000","last_clean":"2024-08-20T06:35:08.973062+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:35:08.973062+0000","last_undegraded":"2024-08-20T06:35:08.973062+0000","last_fullsized":"2024-08-20T06:35:08.973062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928337+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:35:00.928337+0000","last_peered":"2024-08-20T06:35:00.928337+0000","last_clean":"2024-08-20T06:35:00.928337+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:35:00.928337+0000","last_undegraded":"2024-08-20T06:35:00.928337+0000","last_fullsized":"2024-08-20T06:35:00.928337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980696+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:35:09.980696+0000","last_peered":"2024-08-20T06:35:09.980696+0000","last_clean":"2024-08-20T06:35:09.980696+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:35:09.980696+0000","last_undegraded":"2024-08-20T06:35:09.980696+0000","last_fullsized":"2024-08-20T06:35:09.980696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925884+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:35:00.925884+0000","last_peered":"2024-08-20T06:35:00.925884+0000","last_clean":"2024-08-20T06:35:00.925884+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:35:00.925884+0000","last_undegraded":"2024-08-20T06:35:00.925884+0000","last_fullsized":"2024-08-20T06:35:00.925884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930386+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:35:00.930386+0000","last_peered":"2024-08-20T06:35:00.930386+0000","last_clean":"2024-08-20T06:35:00.930386+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:35:00.930386+0000","last_undegraded":"2024-08-20T06:35:00.930386+0000","last_fullsized":"2024-08-20T06:35:00.930386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925856+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:35:00.925856+0000","last_peered":"2024-08-20T06:35:00.925856+0000","last_clean":"2024-08-20T06:35:00.925856+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:35:00.925856+0000","last_undegraded":"2024-08-20T06:35:00.925856+0000","last_fullsized":"2024-08-20T06:35:00.925856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.931810+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:35:00.931810+0000","last_peered":"2024-08-20T06:35:00.931810+0000","last_clean":"2024-08-20T06:35:00.931810+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:35:00.931810+0000","last_undegraded":"2024-08-20T06:35:00.931810+0000","last_fullsized":"2024-08-20T06:35:00.931810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.110553+0000","last_change":"2024-08-20T06:34:59.935512+0000","last_active":"2024-08-20T06:35:02.110553+0000","last_peered":"2024-08-20T06:35:02.110553+0000","last_clean":"2024-08-20T06:35:02.110553+0000","last_became_active":"2024-08-20T06:34:59.933714+0000","last_became_peered":"2024-08-20T06:34:59.933714+0000","last_unstale":"2024-08-20T06:35:02.110553+0000","last_undegraded":"2024-08-20T06:35:02.110553+0000","last_fullsized":"2024-08-20T06:35:02.110553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:55:50.042593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972539+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:35:08.972539+0000","last_peered":"2024-08-20T06:35:08.972539+0000","last_clean":"2024-08-20T06:35:08.972539+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:35:08.972539+0000","last_undegraded":"2024-08-20T06:35:08.972539+0000","last_fullsized":"2024-08-20T06:35:08.972539+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927132+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:35:00.927132+0000","last_peered":"2024-08-20T06:35:00.927132+0000","last_clean":"2024-08-20T06:35:00.927132+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:35:00.927132+0000","last_undegraded":"2024-08-20T06:35:00.927132+0000","last_fullsized":"2024-08-20T06:35:00.927132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924980+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:35:00.924980+0000","last_peered":"2024-08-20T06:35:00.924980+0000","last_clean":"2024-08-20T06:35:00.924980+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:35:00.924980+0000","last_undegraded":"2024-08-20T06:35:00.924980+0000","last_fullsized":"2024-08-20T06:35:00.924980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980820+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:35:09.980820+0000","last_peered":"2024-08-20T06:35:09.980820+0000","last_clean":"2024-08-20T06:35:09.980820+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:35:09.980820+0000","last_undegraded":"2024-08-20T06:35:09.980820+0000","last_fullsized":"2024-08-20T06:35:09.980820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925610+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:35:00.925610+0000","last_peered":"2024-08-20T06:35:00.925610+0000","last_clean":"2024-08-20T06:35:00.925610+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:35:00.925610+0000","last_undegraded":"2024-08-20T06:35:00.925610+0000","last_fullsized":"2024-08-20T06:35:00.925610+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972913+0000","last_change":"2024-08-20T06:34:57.914079+0000","last_active":"2024-08-20T06:35:08.972913+0000","last_peered":"2024-08-20T06:35:08.972913+0000","last_clean":"2024-08-20T06:35:08.972913+0000","last_became_active":"2024-08-20T06:34:57.913747+0000","last_became_peered":"2024-08-20T06:34:57.913747+0000","last_unstale":"2024-08-20T06:35:08.972913+0000","last_undegraded":"2024-08-20T06:35:08.972913+0000","last_fullsized":"2024-08-20T06:35:08.972913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:21:24.138487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.923909+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:35:00.923909+0000","last_peered":"2024-08-20T06:35:00.923909+0000","last_clean":"2024-08-20T06:35:00.923909+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:35:00.923909+0000","last_undegraded":"2024-08-20T06:35:00.923909+0000","last_fullsized":"2024-08-20T06:35:00.923909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:21.001265+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:35:21.001265+0000","last_peered":"2024-08-20T06:35:21.001265+0000","last_clean":"2024-08-20T06:35:21.001265+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:35:21.001265+0000","last_undegraded":"2024-08-20T06:35:21.001265+0000","last_fullsized":"2024-08-20T06:35:21.001265+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928003+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:35:00.928003+0000","last_peered":"2024-08-20T06:35:00.928003+0000","last_clean":"2024-08-20T06:35:00.928003+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:35:00.928003+0000","last_undegraded":"2024-08-20T06:35:00.928003+0000","last_fullsized":"2024-08-20T06:35:00.928003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925692+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:35:00.925692+0000","last_peered":"2024-08-20T06:35:00.925692+0000","last_clean":"2024-08-20T06:35:00.925692+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:35:00.925692+0000","last_undegraded":"2024-08-20T06:35:00.925692+0000","last_fullsized":"2024-08-20T06:35:00.925692+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924540+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:35:00.924540+0000","last_peered":"2024-08-20T06:35:00.924540+0000","last_clean":"2024-08-20T06:35:00.924540+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:35:00.924540+0000","last_undegraded":"2024-08-20T06:35:00.924540+0000","last_fullsized":"2024-08-20T06:35:00.924540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916869+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:35:00.916869+0000","last_peered":"2024-08-20T06:35:00.916869+0000","last_clean":"2024-08-20T06:35:00.916869+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:35:00.916869+0000","last_undegraded":"2024-08-20T06:35:00.916869+0000","last_fullsized":"2024-08-20T06:35:00.916869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.267391+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:35:02.267391+0000","last_peered":"2024-08-20T06:35:02.267391+0000","last_clean":"2024-08-20T06:35:02.267391+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:35:02.267391+0000","last_undegraded":"2024-08-20T06:35:02.267391+0000","last_fullsized":"2024-08-20T06:35:02.267391+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965120+0000","last_change":"2024-08-20T06:34:57.973505+0000","last_active":"2024-08-20T06:35:07.965120+0000","last_peered":"2024-08-20T06:35:07.965120+0000","last_clean":"2024-08-20T06:35:07.965120+0000","last_became_active":"2024-08-20T06:34:57.973120+0000","last_became_peered":"2024-08-20T06:34:57.973120+0000","last_unstale":"2024-08-20T06:35:07.965120+0000","last_undegraded":"2024-08-20T06:35:07.965120+0000","last_fullsized":"2024-08-20T06:35:07.965120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:29:02.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926155+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:35:00.926155+0000","last_peered":"2024-08-20T06:35:00.926155+0000","last_clean":"2024-08-20T06:35:00.926155+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:35:00.926155+0000","last_undegraded":"2024-08-20T06:35:00.926155+0000","last_fullsized":"2024-08-20T06:35:00.926155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'6","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:15.923220+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:35:15.923220+0000","last_peered":"2024-08-20T06:35:15.923220+0000","last_clean":"2024-08-20T06:35:15.923220+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:35:15.923220+0000","last_undegraded":"2024-08-20T06:35:15.923220+0000","last_fullsized":"2024-08-20T06:35:15.923220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925443+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:35:00.925443+0000","last_peered":"2024-08-20T06:35:00.925443+0000","last_clean":"2024-08-20T06:35:00.925443+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:35:00.925443+0000","last_undegraded":"2024-08-20T06:35:00.925443+0000","last_fullsized":"2024-08-20T06:35:00.925443+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964887+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:35:07.964887+0000","last_peered":"2024-08-20T06:35:07.964887+0000","last_clean":"2024-08-20T06:35:07.964887+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:35:07.964887+0000","last_undegraded":"2024-08-20T06:35:07.964887+0000","last_fullsized":"2024-08-20T06:35:07.964887+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927968+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:35:00.927968+0000","last_peered":"2024-08-20T06:35:00.927968+0000","last_clean":"2024-08-20T06:35:00.927968+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:35:00.927968+0000","last_undegraded":"2024-08-20T06:35:00.927968+0000","last_fullsized":"2024-08-20T06:35:00.927968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964142+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:35:07.964142+0000","last_peered":"2024-08-20T06:35:07.964142+0000","last_clean":"2024-08-20T06:35:07.964142+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:35:07.964142+0000","last_undegraded":"2024-08-20T06:35:07.964142+0000","last_fullsized":"2024-08-20T06:35:07.964142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924239+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:35:00.924239+0000","last_peered":"2024-08-20T06:35:00.924239+0000","last_clean":"2024-08-20T06:35:00.924239+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:35:00.924239+0000","last_undegraded":"2024-08-20T06:35:00.924239+0000","last_fullsized":"2024-08-20T06:35:00.924239+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965180+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:35:07.965180+0000","last_peered":"2024-08-20T06:35:07.965180+0000","last_clean":"2024-08-20T06:35:07.965180+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:35:07.965180+0000","last_undegraded":"2024-08-20T06:35:07.965180+0000","last_fullsized":"2024-08-20T06:35:07.965180+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924929+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:35:00.924929+0000","last_peered":"2024-08-20T06:35:00.924929+0000","last_clean":"2024-08-20T06:35:00.924929+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:35:00.924929+0000","last_undegraded":"2024-08-20T06:35:00.924929+0000","last_fullsized":"2024-08-20T06:35:00.924929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917681+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:35:00.917681+0000","last_peered":"2024-08-20T06:35:00.917681+0000","last_clean":"2024-08-20T06:35:00.917681+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:35:00.917681+0000","last_undegraded":"2024-08-20T06:35:00.917681+0000","last_fullsized":"2024-08-20T06:35:00.917681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925386+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:35:00.925386+0000","last_peered":"2024-08-20T06:35:00.925386+0000","last_clean":"2024-08-20T06:35:00.925386+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:35:00.925386+0000","last_undegraded":"2024-08-20T06:35:00.925386+0000","last_fullsized":"2024-08-20T06:35:00.925386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965620+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:35:07.965620+0000","last_peered":"2024-08-20T06:35:07.965620+0000","last_clean":"2024-08-20T06:35:07.965620+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:35:07.965620+0000","last_undegraded":"2024-08-20T06:35:07.965620+0000","last_fullsized":"2024-08-20T06:35:07.965620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928848+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:35:00.928848+0000","last_peered":"2024-08-20T06:35:00.928848+0000","last_clean":"2024-08-20T06:35:00.928848+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:35:00.928848+0000","last_undegraded":"2024-08-20T06:35:00.928848+0000","last_fullsized":"2024-08-20T06:35:00.928848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928496+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:35:00.928496+0000","last_peered":"2024-08-20T06:35:00.928496+0000","last_clean":"2024-08-20T06:35:00.928496+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:35:00.928496+0000","last_undegraded":"2024-08-20T06:35:00.928496+0000","last_fullsized":"2024-08-20T06:35:00.928496+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930358+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:35:00.930358+0000","last_peered":"2024-08-20T06:35:00.930358+0000","last_clean":"2024-08-20T06:35:00.930358+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:35:00.930358+0000","last_undegraded":"2024-08-20T06:35:00.930358+0000","last_fullsized":"2024-08-20T06:35:00.930358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928618+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:35:00.928618+0000","last_peered":"2024-08-20T06:35:00.928618+0000","last_clean":"2024-08-20T06:35:00.928618+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:35:00.928618+0000","last_undegraded":"2024-08-20T06:35:00.928618+0000","last_fullsized":"2024-08-20T06:35:00.928618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971790+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:35:08.971790+0000","last_peered":"2024-08-20T06:35:08.971790+0000","last_clean":"2024-08-20T06:35:08.971790+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:35:08.971790+0000","last_undegraded":"2024-08-20T06:35:08.971790+0000","last_fullsized":"2024-08-20T06:35:08.971790+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928116+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:35:00.928116+0000","last_peered":"2024-08-20T06:35:00.928116+0000","last_clean":"2024-08-20T06:35:00.928116+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:35:00.928116+0000","last_undegraded":"2024-08-20T06:35:00.928116+0000","last_fullsized":"2024-08-20T06:35:00.928116+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930857+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:35:00.930857+0000","last_peered":"2024-08-20T06:35:00.930857+0000","last_clean":"2024-08-20T06:35:00.930857+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:35:00.930857+0000","last_undegraded":"2024-08-20T06:35:00.930857+0000","last_fullsized":"2024-08-20T06:35:00.930857+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925854+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:35:00.925854+0000","last_peered":"2024-08-20T06:35:00.925854+0000","last_clean":"2024-08-20T06:35:00.925854+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:35:00.925854+0000","last_undegraded":"2024-08-20T06:35:00.925854+0000","last_fullsized":"2024-08-20T06:35:00.925854+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973139+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:35:08.973139+0000","last_peered":"2024-08-20T06:35:08.973139+0000","last_clean":"2024-08-20T06:35:08.973139+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:35:08.973139+0000","last_undegraded":"2024-08-20T06:35:08.973139+0000","last_fullsized":"2024-08-20T06:35:08.973139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55708,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":101,"num_read_kb":110,"num_write":149,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":145054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717380,"statfs":{"total":95995035648,"available":95966597120,"internally_reserved":0,"allocated":765952,"data_stored":368498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1454080,"data_stored":1012077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168602,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528356,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1376256,"data_stored":960193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724584,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27776,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":770048,"data_stored":368123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888109,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28380,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716772,"statfs":{"total":95995035648,"available":95965974528,"internally_reserved":0,"allocated":1388544,"data_stored":959877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051635,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713212,"statfs":{"total":95995035648,"available":95962329088,"internally_reserved":0,"allocated":774144,"data_stored":368560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":368434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444092,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36160,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708992,"statfs":{"total":95995035648,"available":95958007808,"internally_reserved":0,"allocated":835584,"data_stored":410329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607614,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":408979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":790528,"data_stored":369394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":41063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:35:25.602 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:35:25.851 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:26.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:26 smithi107 ceph-mon[18495]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:26.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:26 smithi193 ceph-mon[21944]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- 172.21.15.107:0/911918136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0102ab0 msgr2=0x7f0be0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 --2- 172.21.15.107:0/911918136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0102ab0 0x7f0be0102e90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0bd0009900 tx=0x7f0bd002a410 comp rx=0 tx=0).stop 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- 172.21.15.107:0/911918136 shutdown_connections 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 --2- 172.21.15.107:0/911918136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be0108e40 0x7f0be010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 --2- 172.21.15.107:0/911918136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be01084c0 0x7f0be0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 --2- 172.21.15.107:0/911918136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0102ab0 0x7f0be0102e90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- 172.21.15.107:0/911918136 >> 172.21.15.107:0/911918136 conn(0x7f0be00fe500 msgr2=0x7f0be0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- 172.21.15.107:0/911918136 shutdown_connections 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- 172.21.15.107:0/911918136 wait complete. 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 Processor -- start 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.319+0000 7f0be6424640 1 -- start start 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0102ab0 0x7f0be00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01084c0 0x7f0be0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0112970 con 0x7f0be0108e40 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0be01127f0 con 0x7f0be01084c0 2024-08-20T06:35:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be6424640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0be0112af0 con 0x7f0be0102ab0 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47606/0 (socket says 172.21.15.107:47606) 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 -- 172.21.15.107:0/434511552 learned_addr learned my addr 172.21.15.107:0/434511552 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0bd7fff640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01084c0 0x7f0be0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0bdffff640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0102ab0 0x7f0be00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 -- 172.21.15.107:0/434511552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0102ab0 msgr2=0x7f0be00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0102ab0 0x7f0be00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 -- 172.21.15.107:0/434511552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01084c0 msgr2=0x7f0be0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01084c0 0x7f0be0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 -- 172.21.15.107:0/434511552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0be01a72c0 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0be49c1640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800eaa0 tx=0x7f0bd800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.320+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd8004d20 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.321+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0bd8005020 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.321+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0be01a75b0 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.321+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd80116d0 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.321+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0be01a7af0 con 0x7f0be0108e40 2024-08-20T06:35:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.322+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0be0102e90 con 0x7f0be0108e40 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.323+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f0bd80051d0 con 0x7f0be0108e40 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.323+0000 7f0bddffb640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 0x7f0bcc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.323+0000 7f0bdffff640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 0x7f0bcc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.324+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f0bd80e3be0 con 0x7f0be0108e40 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.324+0000 7f0bdffff640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 0x7f0bcc0b61a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00057c0 tx=0x7f0bd00058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.327+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0bd8016030 con 0x7f0be0108e40 2024-08-20T06:35:26.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:26 smithi183 ceph-mon[22204]: from='client.14901 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:26.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.499+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0be00632c0 con 0x7f0bcc0b3ce0 2024-08-20T06:35:26.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.503+0000 7f0bddffb640 1 -- 172.21.15.107:0/434511552 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258689 (secure 0 0 0) 0x7f0be00632c0 con 0x7f0bcc0b3ce0 2024-08-20T06:35:26.506 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:26.507 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 msgr2=0x7f0bcc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 0x7f0bcc0b61a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00057c0 tx=0x7f0bd00058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 msgr2=0x7f0be0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bd800eaa0 tx=0x7f0bd800eea0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 shutdown_connections 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0be0108e40 0x7f0be0075c50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0be01084c0 0x7f0be0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0bcc0b3ce0 0x7f0bcc0b61a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 --2- 172.21.15.107:0/434511552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0be0102ab0 0x7f0be00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:26.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 >> 172.21.15.107:0/434511552 conn(0x7f0be00fe500 msgr2=0x7f0be01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:26.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 shutdown_connections 2024-08-20T06:35:26.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:26.506+0000 7f0be6424640 1 -- 172.21.15.107:0/434511552 wait complete. 2024-08-20T06:35:27.236 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:27 smithi107 ceph-mon[18495]: pgmap v283: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-08-20T06:35:27.266 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-08-20T06:35:25.311059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":646310,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":182,"num_read_kb":174,"num_write":274,"num_write_kb":2248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":224,"ondisk_log_size":224,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364484,"kb_used_data":11076,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577340,"statfs":{"total":1151940427776,"available":1151567196160,"internally_reserved":0,"allocated":11341824,"data_stored":6353922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":50720,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":83,"num_write":26,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926320+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:35:00.926320+0000","last_peered":"2024-08-20T06:35:00.926320+0000","last_clean":"2024-08-20T06:35:00.926320+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:35:00.926320+0000","last_undegraded":"2024-08-20T06:35:00.926320+0000","last_fullsized":"2024-08-20T06:35:00.926320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927883+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:35:00.927883+0000","last_peered":"2024-08-20T06:35:00.927883+0000","last_clean":"2024-08-20T06:35:00.927883+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:35:00.927883+0000","last_undegraded":"2024-08-20T06:35:00.927883+0000","last_fullsized":"2024-08-20T06:35:00.927883+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927658+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:35:00.927658+0000","last_peered":"2024-08-20T06:35:00.927658+0000","last_clean":"2024-08-20T06:35:00.927658+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:35:00.927658+0000","last_undegraded":"2024-08-20T06:35:00.927658+0000","last_fullsized":"2024-08-20T06:35:00.927658+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918161+0000","last_change":"2024-08-20T06:34:59.933963+0000","last_active":"2024-08-20T06:35:00.918161+0000","last_peered":"2024-08-20T06:35:00.918161+0000","last_clean":"2024-08-20T06:35:00.918161+0000","last_became_active":"2024-08-20T06:34:59.933682+0000","last_became_peered":"2024-08-20T06:34:59.933682+0000","last_unstale":"2024-08-20T06:35:00.918161+0000","last_undegraded":"2024-08-20T06:35:00.918161+0000","last_fullsized":"2024-08-20T06:35:00.918161+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:07.550290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924480+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:35:00.924480+0000","last_peered":"2024-08-20T06:35:00.924480+0000","last_clean":"2024-08-20T06:35:00.924480+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:35:00.924480+0000","last_undegraded":"2024-08-20T06:35:00.924480+0000","last_fullsized":"2024-08-20T06:35:00.924480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924408+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:35:00.924408+0000","last_peered":"2024-08-20T06:35:00.924408+0000","last_clean":"2024-08-20T06:35:00.924408+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:35:00.924408+0000","last_undegraded":"2024-08-20T06:35:00.924408+0000","last_fullsized":"2024-08-20T06:35:00.924408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925543+0000","last_change":"2024-08-20T06:34:59.933789+0000","last_active":"2024-08-20T06:35:00.925543+0000","last_peered":"2024-08-20T06:35:00.925543+0000","last_clean":"2024-08-20T06:35:00.925543+0000","last_became_active":"2024-08-20T06:34:59.933461+0000","last_became_peered":"2024-08-20T06:34:59.933461+0000","last_unstale":"2024-08-20T06:35:00.925543+0000","last_undegraded":"2024-08-20T06:35:00.925543+0000","last_fullsized":"2024-08-20T06:35:00.925543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:31:37.924168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924924+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:35:00.924924+0000","last_peered":"2024-08-20T06:35:00.924924+0000","last_clean":"2024-08-20T06:35:00.924924+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:35:00.924924+0000","last_undegraded":"2024-08-20T06:35:00.924924+0000","last_fullsized":"2024-08-20T06:35:00.924924+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928568+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:35:00.928568+0000","last_peered":"2024-08-20T06:35:00.928568+0000","last_clean":"2024-08-20T06:35:00.928568+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:35:00.928568+0000","last_undegraded":"2024-08-20T06:35:00.928568+0000","last_fullsized":"2024-08-20T06:35:00.928568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925563+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:35:00.925563+0000","last_peered":"2024-08-20T06:35:00.925563+0000","last_clean":"2024-08-20T06:35:00.925563+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:35:00.925563+0000","last_undegraded":"2024-08-20T06:35:00.925563+0000","last_fullsized":"2024-08-20T06:35:00.925563+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926964+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:35:00.926964+0000","last_peered":"2024-08-20T06:35:00.926964+0000","last_clean":"2024-08-20T06:35:00.926964+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:35:00.926964+0000","last_undegraded":"2024-08-20T06:35:00.926964+0000","last_fullsized":"2024-08-20T06:35:00.926964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918213+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:35:00.918213+0000","last_peered":"2024-08-20T06:35:00.918213+0000","last_clean":"2024-08-20T06:35:00.918213+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:35:00.918213+0000","last_undegraded":"2024-08-20T06:35:00.918213+0000","last_fullsized":"2024-08-20T06:35:00.918213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924444+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:35:00.924444+0000","last_peered":"2024-08-20T06:35:00.924444+0000","last_clean":"2024-08-20T06:35:00.924444+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:35:00.924444+0000","last_undegraded":"2024-08-20T06:35:00.924444+0000","last_fullsized":"2024-08-20T06:35:00.924444+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927989+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:35:00.927989+0000","last_peered":"2024-08-20T06:35:00.927989+0000","last_clean":"2024-08-20T06:35:00.927989+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:35:00.927989+0000","last_undegraded":"2024-08-20T06:35:00.927989+0000","last_fullsized":"2024-08-20T06:35:00.927989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927600+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:35:00.927600+0000","last_peered":"2024-08-20T06:35:00.927600+0000","last_clean":"2024-08-20T06:35:00.927600+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:35:00.927600+0000","last_undegraded":"2024-08-20T06:35:00.927600+0000","last_fullsized":"2024-08-20T06:35:00.927600+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928257+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:35:00.928257+0000","last_peered":"2024-08-20T06:35:00.928257+0000","last_clean":"2024-08-20T06:35:00.928257+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:35:00.928257+0000","last_undegraded":"2024-08-20T06:35:00.928257+0000","last_fullsized":"2024-08-20T06:35:00.928257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926363+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:35:00.926363+0000","last_peered":"2024-08-20T06:35:00.926363+0000","last_clean":"2024-08-20T06:35:00.926363+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:35:00.926363+0000","last_undegraded":"2024-08-20T06:35:00.926363+0000","last_fullsized":"2024-08-20T06:35:00.926363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917047+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:35:00.917047+0000","last_peered":"2024-08-20T06:35:00.917047+0000","last_clean":"2024-08-20T06:35:00.917047+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:35:00.917047+0000","last_undegraded":"2024-08-20T06:35:00.917047+0000","last_fullsized":"2024-08-20T06:35:00.917047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917498+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:35:00.917498+0000","last_peered":"2024-08-20T06:35:00.917498+0000","last_clean":"2024-08-20T06:35:00.917498+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:35:00.917498+0000","last_undegraded":"2024-08-20T06:35:00.917498+0000","last_fullsized":"2024-08-20T06:35:00.917498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924758+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:35:00.924758+0000","last_peered":"2024-08-20T06:35:00.924758+0000","last_clean":"2024-08-20T06:35:00.924758+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:35:00.924758+0000","last_undegraded":"2024-08-20T06:35:00.924758+0000","last_fullsized":"2024-08-20T06:35:00.924758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924606+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:35:00.924606+0000","last_peered":"2024-08-20T06:35:00.924606+0000","last_clean":"2024-08-20T06:35:00.924606+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:35:00.924606+0000","last_undegraded":"2024-08-20T06:35:00.924606+0000","last_fullsized":"2024-08-20T06:35:00.924606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928457+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:35:00.928457+0000","last_peered":"2024-08-20T06:35:00.928457+0000","last_clean":"2024-08-20T06:35:00.928457+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:35:00.928457+0000","last_undegraded":"2024-08-20T06:35:00.928457+0000","last_fullsized":"2024-08-20T06:35:00.928457+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924874+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:35:00.924874+0000","last_peered":"2024-08-20T06:35:00.924874+0000","last_clean":"2024-08-20T06:35:00.924874+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:35:00.924874+0000","last_undegraded":"2024-08-20T06:35:00.924874+0000","last_fullsized":"2024-08-20T06:35:00.924874+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927847+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:35:00.927847+0000","last_peered":"2024-08-20T06:35:00.927847+0000","last_clean":"2024-08-20T06:35:00.927847+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:35:00.927847+0000","last_undegraded":"2024-08-20T06:35:00.927847+0000","last_fullsized":"2024-08-20T06:35:00.927847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925294+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:35:00.925294+0000","last_peered":"2024-08-20T06:35:00.925294+0000","last_clean":"2024-08-20T06:35:00.925294+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:35:00.925294+0000","last_undegraded":"2024-08-20T06:35:00.925294+0000","last_fullsized":"2024-08-20T06:35:00.925294+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917752+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:35:00.917752+0000","last_peered":"2024-08-20T06:35:00.917752+0000","last_clean":"2024-08-20T06:35:00.917752+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:35:00.917752+0000","last_undegraded":"2024-08-20T06:35:00.917752+0000","last_fullsized":"2024-08-20T06:35:00.917752+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925219+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:35:00.925219+0000","last_peered":"2024-08-20T06:35:00.925219+0000","last_clean":"2024-08-20T06:35:00.925219+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:35:00.925219+0000","last_undegraded":"2024-08-20T06:35:00.925219+0000","last_fullsized":"2024-08-20T06:35:00.925219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925939+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:35:00.925939+0000","last_peered":"2024-08-20T06:35:00.925939+0000","last_clean":"2024-08-20T06:35:00.925939+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:35:00.925939+0000","last_undegraded":"2024-08-20T06:35:00.925939+0000","last_fullsized":"2024-08-20T06:35:00.925939+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928092+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:35:00.928092+0000","last_peered":"2024-08-20T06:35:00.928092+0000","last_clean":"2024-08-20T06:35:00.928092+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:35:00.928092+0000","last_undegraded":"2024-08-20T06:35:00.928092+0000","last_fullsized":"2024-08-20T06:35:00.928092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917181+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:35:00.917181+0000","last_peered":"2024-08-20T06:35:00.917181+0000","last_clean":"2024-08-20T06:35:00.917181+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:35:00.917181+0000","last_undegraded":"2024-08-20T06:35:00.917181+0000","last_fullsized":"2024-08-20T06:35:00.917181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917419+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:35:00.917419+0000","last_peered":"2024-08-20T06:35:00.917419+0000","last_clean":"2024-08-20T06:35:00.917419+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:35:00.917419+0000","last_undegraded":"2024-08-20T06:35:00.917419+0000","last_fullsized":"2024-08-20T06:35:00.917419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924729+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:35:00.924729+0000","last_peered":"2024-08-20T06:35:00.924729+0000","last_clean":"2024-08-20T06:35:00.924729+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:35:00.924729+0000","last_undegraded":"2024-08-20T06:35:00.924729+0000","last_fullsized":"2024-08-20T06:35:00.924729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924280+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:35:00.924280+0000","last_peered":"2024-08-20T06:35:00.924280+0000","last_clean":"2024-08-20T06:35:00.924280+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:35:00.924280+0000","last_undegraded":"2024-08-20T06:35:00.924280+0000","last_fullsized":"2024-08-20T06:35:00.924280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:24.006395+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:35:24.006395+0000","last_peered":"2024-08-20T06:35:24.006395+0000","last_clean":"2024-08-20T06:35:24.006395+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:35:24.006395+0000","last_undegraded":"2024-08-20T06:35:24.006395+0000","last_fullsized":"2024-08-20T06:35:24.006395+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928023+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:35:00.928023+0000","last_peered":"2024-08-20T06:35:00.928023+0000","last_clean":"2024-08-20T06:35:00.928023+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:35:00.928023+0000","last_undegraded":"2024-08-20T06:35:00.928023+0000","last_fullsized":"2024-08-20T06:35:00.928023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973053+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:35:08.973053+0000","last_peered":"2024-08-20T06:35:08.973053+0000","last_clean":"2024-08-20T06:35:08.973053+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:35:08.973053+0000","last_undegraded":"2024-08-20T06:35:08.973053+0000","last_fullsized":"2024-08-20T06:35:08.973053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971914+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:35:08.971914+0000","last_peered":"2024-08-20T06:35:08.971914+0000","last_clean":"2024-08-20T06:35:08.971914+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:35:08.971914+0000","last_undegraded":"2024-08-20T06:35:08.971914+0000","last_fullsized":"2024-08-20T06:35:08.971914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925068+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:35:00.925068+0000","last_peered":"2024-08-20T06:35:00.925068+0000","last_clean":"2024-08-20T06:35:00.925068+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:35:00.925068+0000","last_undegraded":"2024-08-20T06:35:00.925068+0000","last_fullsized":"2024-08-20T06:35:00.925068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928244+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:35:00.928244+0000","last_peered":"2024-08-20T06:35:00.928244+0000","last_clean":"2024-08-20T06:35:00.928244+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:35:00.928244+0000","last_undegraded":"2024-08-20T06:35:00.928244+0000","last_fullsized":"2024-08-20T06:35:00.928244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916637+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:35:00.916637+0000","last_peered":"2024-08-20T06:35:00.916637+0000","last_clean":"2024-08-20T06:35:00.916637+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:35:00.916637+0000","last_undegraded":"2024-08-20T06:35:00.916637+0000","last_fullsized":"2024-08-20T06:35:00.916637+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:21.000990+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:35:21.000990+0000","last_peered":"2024-08-20T06:35:21.000990+0000","last_clean":"2024-08-20T06:35:21.000990+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:35:21.000990+0000","last_undegraded":"2024-08-20T06:35:21.000990+0000","last_fullsized":"2024-08-20T06:35:21.000990+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928449+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:35:00.928449+0000","last_peered":"2024-08-20T06:35:00.928449+0000","last_clean":"2024-08-20T06:35:00.928449+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:35:00.928449+0000","last_undegraded":"2024-08-20T06:35:00.928449+0000","last_fullsized":"2024-08-20T06:35:00.928449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965368+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:35:07.965368+0000","last_peered":"2024-08-20T06:35:07.965368+0000","last_clean":"2024-08-20T06:35:07.965368+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:35:07.965368+0000","last_undegraded":"2024-08-20T06:35:07.965368+0000","last_fullsized":"2024-08-20T06:35:07.965368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928137+0000","last_change":"2024-08-20T06:34:59.930088+0000","last_active":"2024-08-20T06:35:00.928137+0000","last_peered":"2024-08-20T06:35:00.928137+0000","last_clean":"2024-08-20T06:35:00.928137+0000","last_became_active":"2024-08-20T06:34:59.929780+0000","last_became_peered":"2024-08-20T06:34:59.929780+0000","last_unstale":"2024-08-20T06:35:00.928137+0000","last_undegraded":"2024-08-20T06:35:00.928137+0000","last_fullsized":"2024-08-20T06:35:00.928137+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:11.617858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964783+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:35:07.964783+0000","last_peered":"2024-08-20T06:35:07.964783+0000","last_clean":"2024-08-20T06:35:07.964783+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:35:07.964783+0000","last_undegraded":"2024-08-20T06:35:07.964783+0000","last_fullsized":"2024-08-20T06:35:07.964783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928799+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:35:00.928799+0000","last_peered":"2024-08-20T06:35:00.928799+0000","last_clean":"2024-08-20T06:35:00.928799+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:35:00.928799+0000","last_undegraded":"2024-08-20T06:35:00.928799+0000","last_fullsized":"2024-08-20T06:35:00.928799+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'26","reported_seq":77,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:22.004860+0000","last_change":"2024-08-20T06:34:57.920527+0000","last_active":"2024-08-20T06:35:22.004860+0000","last_peered":"2024-08-20T06:35:22.004860+0000","last_clean":"2024-08-20T06:35:22.004860+0000","last_became_active":"2024-08-20T06:34:57.920116+0000","last_became_peered":"2024-08-20T06:34:57.920116+0000","last_unstale":"2024-08-20T06:35:22.004860+0000","last_undegraded":"2024-08-20T06:35:22.004860+0000","last_fullsized":"2024-08-20T06:35:22.004860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:46:06.121047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52098,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":66,"num_write":27,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926259+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:35:00.926259+0000","last_peered":"2024-08-20T06:35:00.926259+0000","last_clean":"2024-08-20T06:35:00.926259+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:35:00.926259+0000","last_undegraded":"2024-08-20T06:35:00.926259+0000","last_fullsized":"2024-08-20T06:35:00.926259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972040+0000","last_change":"2024-08-20T06:34:57.919137+0000","last_active":"2024-08-20T06:35:08.972040+0000","last_peered":"2024-08-20T06:35:08.972040+0000","last_clean":"2024-08-20T06:35:08.972040+0000","last_became_active":"2024-08-20T06:34:57.918821+0000","last_became_peered":"2024-08-20T06:34:57.918821+0000","last_unstale":"2024-08-20T06:35:08.972040+0000","last_undegraded":"2024-08-20T06:35:08.972040+0000","last_fullsized":"2024-08-20T06:35:08.972040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:51:18.267609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.923922+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:35:00.923922+0000","last_peered":"2024-08-20T06:35:00.923922+0000","last_clean":"2024-08-20T06:35:00.923922+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:35:00.923922+0000","last_undegraded":"2024-08-20T06:35:00.923922+0000","last_fullsized":"2024-08-20T06:35:00.923922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964646+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:35:07.964646+0000","last_peered":"2024-08-20T06:35:07.964646+0000","last_clean":"2024-08-20T06:35:07.964646+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:35:07.964646+0000","last_undegraded":"2024-08-20T06:35:07.964646+0000","last_fullsized":"2024-08-20T06:35:07.964646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925098+0000","last_change":"2024-08-20T06:34:59.930132+0000","last_active":"2024-08-20T06:35:00.925098+0000","last_peered":"2024-08-20T06:35:00.925098+0000","last_clean":"2024-08-20T06:35:00.925098+0000","last_became_active":"2024-08-20T06:34:59.929820+0000","last_became_peered":"2024-08-20T06:34:59.929820+0000","last_unstale":"2024-08-20T06:35:00.925098+0000","last_undegraded":"2024-08-20T06:35:00.925098+0000","last_fullsized":"2024-08-20T06:35:00.925098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:30.947526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":38,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:22.002972+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:35:22.002972+0000","last_peered":"2024-08-20T06:35:22.002972+0000","last_clean":"2024-08-20T06:35:22.002972+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:35:22.002972+0000","last_undegraded":"2024-08-20T06:35:22.002972+0000","last_fullsized":"2024-08-20T06:35:22.002972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":17,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924673+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:35:00.924673+0000","last_peered":"2024-08-20T06:35:00.924673+0000","last_clean":"2024-08-20T06:35:00.924673+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:35:00.924673+0000","last_undegraded":"2024-08-20T06:35:00.924673+0000","last_fullsized":"2024-08-20T06:35:00.924673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973062+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:35:08.973062+0000","last_peered":"2024-08-20T06:35:08.973062+0000","last_clean":"2024-08-20T06:35:08.973062+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:35:08.973062+0000","last_undegraded":"2024-08-20T06:35:08.973062+0000","last_fullsized":"2024-08-20T06:35:08.973062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928337+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:35:00.928337+0000","last_peered":"2024-08-20T06:35:00.928337+0000","last_clean":"2024-08-20T06:35:00.928337+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:35:00.928337+0000","last_undegraded":"2024-08-20T06:35:00.928337+0000","last_fullsized":"2024-08-20T06:35:00.928337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980696+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:35:09.980696+0000","last_peered":"2024-08-20T06:35:09.980696+0000","last_clean":"2024-08-20T06:35:09.980696+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:35:09.980696+0000","last_undegraded":"2024-08-20T06:35:09.980696+0000","last_fullsized":"2024-08-20T06:35:09.980696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925884+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:35:00.925884+0000","last_peered":"2024-08-20T06:35:00.925884+0000","last_clean":"2024-08-20T06:35:00.925884+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:35:00.925884+0000","last_undegraded":"2024-08-20T06:35:00.925884+0000","last_fullsized":"2024-08-20T06:35:00.925884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930386+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:35:00.930386+0000","last_peered":"2024-08-20T06:35:00.930386+0000","last_clean":"2024-08-20T06:35:00.930386+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:35:00.930386+0000","last_undegraded":"2024-08-20T06:35:00.930386+0000","last_fullsized":"2024-08-20T06:35:00.930386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925856+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:35:00.925856+0000","last_peered":"2024-08-20T06:35:00.925856+0000","last_clean":"2024-08-20T06:35:00.925856+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:35:00.925856+0000","last_undegraded":"2024-08-20T06:35:00.925856+0000","last_fullsized":"2024-08-20T06:35:00.925856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.931810+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:35:00.931810+0000","last_peered":"2024-08-20T06:35:00.931810+0000","last_clean":"2024-08-20T06:35:00.931810+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:35:00.931810+0000","last_undegraded":"2024-08-20T06:35:00.931810+0000","last_fullsized":"2024-08-20T06:35:00.931810+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.110553+0000","last_change":"2024-08-20T06:34:59.935512+0000","last_active":"2024-08-20T06:35:02.110553+0000","last_peered":"2024-08-20T06:35:02.110553+0000","last_clean":"2024-08-20T06:35:02.110553+0000","last_became_active":"2024-08-20T06:34:59.933714+0000","last_became_peered":"2024-08-20T06:34:59.933714+0000","last_unstale":"2024-08-20T06:35:02.110553+0000","last_undegraded":"2024-08-20T06:35:02.110553+0000","last_fullsized":"2024-08-20T06:35:02.110553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:55:50.042593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972539+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:35:08.972539+0000","last_peered":"2024-08-20T06:35:08.972539+0000","last_clean":"2024-08-20T06:35:08.972539+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:35:08.972539+0000","last_undegraded":"2024-08-20T06:35:08.972539+0000","last_fullsized":"2024-08-20T06:35:08.972539+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927132+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:35:00.927132+0000","last_peered":"2024-08-20T06:35:00.927132+0000","last_clean":"2024-08-20T06:35:00.927132+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:35:00.927132+0000","last_undegraded":"2024-08-20T06:35:00.927132+0000","last_fullsized":"2024-08-20T06:35:00.927132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924980+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:35:00.924980+0000","last_peered":"2024-08-20T06:35:00.924980+0000","last_clean":"2024-08-20T06:35:00.924980+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:35:00.924980+0000","last_undegraded":"2024-08-20T06:35:00.924980+0000","last_fullsized":"2024-08-20T06:35:00.924980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980820+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:35:09.980820+0000","last_peered":"2024-08-20T06:35:09.980820+0000","last_clean":"2024-08-20T06:35:09.980820+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:35:09.980820+0000","last_undegraded":"2024-08-20T06:35:09.980820+0000","last_fullsized":"2024-08-20T06:35:09.980820+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925610+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:35:00.925610+0000","last_peered":"2024-08-20T06:35:00.925610+0000","last_clean":"2024-08-20T06:35:00.925610+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:35:00.925610+0000","last_undegraded":"2024-08-20T06:35:00.925610+0000","last_fullsized":"2024-08-20T06:35:00.925610+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972913+0000","last_change":"2024-08-20T06:34:57.914079+0000","last_active":"2024-08-20T06:35:08.972913+0000","last_peered":"2024-08-20T06:35:08.972913+0000","last_clean":"2024-08-20T06:35:08.972913+0000","last_became_active":"2024-08-20T06:34:57.913747+0000","last_became_peered":"2024-08-20T06:34:57.913747+0000","last_unstale":"2024-08-20T06:35:08.972913+0000","last_undegraded":"2024-08-20T06:35:08.972913+0000","last_fullsized":"2024-08-20T06:35:08.972913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:21:24.138487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.923909+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:35:00.923909+0000","last_peered":"2024-08-20T06:35:00.923909+0000","last_clean":"2024-08-20T06:35:00.923909+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:35:00.923909+0000","last_undegraded":"2024-08-20T06:35:00.923909+0000","last_fullsized":"2024-08-20T06:35:00.923909+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:21.001265+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:35:21.001265+0000","last_peered":"2024-08-20T06:35:21.001265+0000","last_clean":"2024-08-20T06:35:21.001265+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:35:21.001265+0000","last_undegraded":"2024-08-20T06:35:21.001265+0000","last_fullsized":"2024-08-20T06:35:21.001265+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928003+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:35:00.928003+0000","last_peered":"2024-08-20T06:35:00.928003+0000","last_clean":"2024-08-20T06:35:00.928003+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:35:00.928003+0000","last_undegraded":"2024-08-20T06:35:00.928003+0000","last_fullsized":"2024-08-20T06:35:00.928003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925692+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:35:00.925692+0000","last_peered":"2024-08-20T06:35:00.925692+0000","last_clean":"2024-08-20T06:35:00.925692+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:35:00.925692+0000","last_undegraded":"2024-08-20T06:35:00.925692+0000","last_fullsized":"2024-08-20T06:35:00.925692+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924540+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:35:00.924540+0000","last_peered":"2024-08-20T06:35:00.924540+0000","last_clean":"2024-08-20T06:35:00.924540+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:35:00.924540+0000","last_undegraded":"2024-08-20T06:35:00.924540+0000","last_fullsized":"2024-08-20T06:35:00.924540+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916869+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:35:00.916869+0000","last_peered":"2024-08-20T06:35:00.916869+0000","last_clean":"2024-08-20T06:35:00.916869+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:35:00.916869+0000","last_undegraded":"2024-08-20T06:35:00.916869+0000","last_fullsized":"2024-08-20T06:35:00.916869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.267391+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:35:02.267391+0000","last_peered":"2024-08-20T06:35:02.267391+0000","last_clean":"2024-08-20T06:35:02.267391+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:35:02.267391+0000","last_undegraded":"2024-08-20T06:35:02.267391+0000","last_fullsized":"2024-08-20T06:35:02.267391+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965120+0000","last_change":"2024-08-20T06:34:57.973505+0000","last_active":"2024-08-20T06:35:07.965120+0000","last_peered":"2024-08-20T06:35:07.965120+0000","last_clean":"2024-08-20T06:35:07.965120+0000","last_became_active":"2024-08-20T06:34:57.973120+0000","last_became_peered":"2024-08-20T06:34:57.973120+0000","last_unstale":"2024-08-20T06:35:07.965120+0000","last_undegraded":"2024-08-20T06:35:07.965120+0000","last_fullsized":"2024-08-20T06:35:07.965120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:29:02.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926155+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:35:00.926155+0000","last_peered":"2024-08-20T06:35:00.926155+0000","last_clean":"2024-08-20T06:35:00.926155+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:35:00.926155+0000","last_undegraded":"2024-08-20T06:35:00.926155+0000","last_fullsized":"2024-08-20T06:35:00.926155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'6","reported_seq":25,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:15.923220+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:35:15.923220+0000","last_peered":"2024-08-20T06:35:15.923220+0000","last_clean":"2024-08-20T06:35:15.923220+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:35:15.923220+0000","last_undegraded":"2024-08-20T06:35:15.923220+0000","last_fullsized":"2024-08-20T06:35:15.923220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925443+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:35:00.925443+0000","last_peered":"2024-08-20T06:35:00.925443+0000","last_clean":"2024-08-20T06:35:00.925443+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:35:00.925443+0000","last_undegraded":"2024-08-20T06:35:00.925443+0000","last_fullsized":"2024-08-20T06:35:00.925443+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964887+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:35:07.964887+0000","last_peered":"2024-08-20T06:35:07.964887+0000","last_clean":"2024-08-20T06:35:07.964887+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:35:07.964887+0000","last_undegraded":"2024-08-20T06:35:07.964887+0000","last_fullsized":"2024-08-20T06:35:07.964887+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927968+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:35:00.927968+0000","last_peered":"2024-08-20T06:35:00.927968+0000","last_clean":"2024-08-20T06:35:00.927968+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:35:00.927968+0000","last_undegraded":"2024-08-20T06:35:00.927968+0000","last_fullsized":"2024-08-20T06:35:00.927968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.964142+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:35:07.964142+0000","last_peered":"2024-08-20T06:35:07.964142+0000","last_clean":"2024-08-20T06:35:07.964142+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:35:07.964142+0000","last_undegraded":"2024-08-20T06:35:07.964142+0000","last_fullsized":"2024-08-20T06:35:07.964142+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924239+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:35:00.924239+0000","last_peered":"2024-08-20T06:35:00.924239+0000","last_clean":"2024-08-20T06:35:00.924239+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:35:00.924239+0000","last_undegraded":"2024-08-20T06:35:00.924239+0000","last_fullsized":"2024-08-20T06:35:00.924239+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965180+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:35:07.965180+0000","last_peered":"2024-08-20T06:35:07.965180+0000","last_clean":"2024-08-20T06:35:07.965180+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:35:07.965180+0000","last_undegraded":"2024-08-20T06:35:07.965180+0000","last_fullsized":"2024-08-20T06:35:07.965180+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924929+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:35:00.924929+0000","last_peered":"2024-08-20T06:35:00.924929+0000","last_clean":"2024-08-20T06:35:00.924929+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:35:00.924929+0000","last_undegraded":"2024-08-20T06:35:00.924929+0000","last_fullsized":"2024-08-20T06:35:00.924929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917681+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:35:00.917681+0000","last_peered":"2024-08-20T06:35:00.917681+0000","last_clean":"2024-08-20T06:35:00.917681+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:35:00.917681+0000","last_undegraded":"2024-08-20T06:35:00.917681+0000","last_fullsized":"2024-08-20T06:35:00.917681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925386+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:35:00.925386+0000","last_peered":"2024-08-20T06:35:00.925386+0000","last_clean":"2024-08-20T06:35:00.925386+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:35:00.925386+0000","last_undegraded":"2024-08-20T06:35:00.925386+0000","last_fullsized":"2024-08-20T06:35:00.925386+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965620+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:35:07.965620+0000","last_peered":"2024-08-20T06:35:07.965620+0000","last_clean":"2024-08-20T06:35:07.965620+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:35:07.965620+0000","last_undegraded":"2024-08-20T06:35:07.965620+0000","last_fullsized":"2024-08-20T06:35:07.965620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928848+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:35:00.928848+0000","last_peered":"2024-08-20T06:35:00.928848+0000","last_clean":"2024-08-20T06:35:00.928848+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:35:00.928848+0000","last_undegraded":"2024-08-20T06:35:00.928848+0000","last_fullsized":"2024-08-20T06:35:00.928848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928496+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:35:00.928496+0000","last_peered":"2024-08-20T06:35:00.928496+0000","last_clean":"2024-08-20T06:35:00.928496+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:35:00.928496+0000","last_undegraded":"2024-08-20T06:35:00.928496+0000","last_fullsized":"2024-08-20T06:35:00.928496+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930358+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:35:00.930358+0000","last_peered":"2024-08-20T06:35:00.930358+0000","last_clean":"2024-08-20T06:35:00.930358+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:35:00.930358+0000","last_undegraded":"2024-08-20T06:35:00.930358+0000","last_fullsized":"2024-08-20T06:35:00.930358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928618+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:35:00.928618+0000","last_peered":"2024-08-20T06:35:00.928618+0000","last_clean":"2024-08-20T06:35:00.928618+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:35:00.928618+0000","last_undegraded":"2024-08-20T06:35:00.928618+0000","last_fullsized":"2024-08-20T06:35:00.928618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971790+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:35:08.971790+0000","last_peered":"2024-08-20T06:35:08.971790+0000","last_clean":"2024-08-20T06:35:08.971790+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:35:08.971790+0000","last_undegraded":"2024-08-20T06:35:08.971790+0000","last_fullsized":"2024-08-20T06:35:08.971790+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928116+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:35:00.928116+0000","last_peered":"2024-08-20T06:35:00.928116+0000","last_clean":"2024-08-20T06:35:00.928116+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:35:00.928116+0000","last_undegraded":"2024-08-20T06:35:00.928116+0000","last_fullsized":"2024-08-20T06:35:00.928116+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930857+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:35:00.930857+0000","last_peered":"2024-08-20T06:35:00.930857+0000","last_clean":"2024-08-20T06:35:00.930857+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:35:00.930857+0000","last_undegraded":"2024-08-20T06:35:00.930857+0000","last_fullsized":"2024-08-20T06:35:00.930857+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925854+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:35:00.925854+0000","last_peered":"2024-08-20T06:35:00.925854+0000","last_clean":"2024-08-20T06:35:00.925854+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:35:00.925854+0000","last_undegraded":"2024-08-20T06:35:00.925854+0000","last_fullsized":"2024-08-20T06:35:00.925854+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973139+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:35:08.973139+0000","last_peered":"2024-08-20T06:35:08.973139+0000","last_clean":"2024-08-20T06:35:08.973139+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:35:08.973139+0000","last_undegraded":"2024-08-20T06:35:08.973139+0000","last_fullsized":"2024-08-20T06:35:08.973139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":55708,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":106,"num_read_kb":110,"num_write":149,"num_write_kb":96,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":167124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717380,"statfs":{"total":95995035648,"available":95966597120,"internally_reserved":0,"allocated":765952,"data_stored":368498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037781,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28444,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716708,"statfs":{"total":95995035648,"available":95965908992,"internally_reserved":0,"allocated":1454080,"data_stored":1012077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168602,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332128,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528356,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1376256,"data_stored":960193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724584,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27776,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717376,"statfs":{"total":95995035648,"available":95966593024,"internally_reserved":0,"allocated":770048,"data_stored":368123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888110,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28380,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716772,"statfs":{"total":95995035648,"available":95965974528,"internally_reserved":0,"allocated":1388544,"data_stored":959877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051635,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713212,"statfs":{"total":95995035648,"available":95962329088,"internally_reserved":0,"allocated":774144,"data_stored":368560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247862,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":368434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444093,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":847872,"data_stored":421364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607615,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":420014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31956,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713196,"statfs":{"total":95995035648,"available":95962312704,"internally_reserved":0,"allocated":790528,"data_stored":369394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":52098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":53676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:35:27.270 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T06:35:27.271 INFO:tasks.cephadm:Setting up client nodes... 2024-08-20T06:35:27.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-20T06:35:27.286 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:27 smithi183 ceph-mon[22204]: pgmap v283: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-08-20T06:35:27.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:27 smithi193 ceph-mon[21944]: pgmap v283: 97 pgs: 97 active+clean; 631 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 5.0 KiB/s wr, 8 op/s 2024-08-20T06:35:27.520 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:27.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 -- 172.21.15.107:0/3789949611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc103890 msgr2=0x7fcebc103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 --2- 172.21.15.107:0/3789949611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc103890 0x7fcebc103d10 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009930 tx=0x7fceb002a420 comp rx=0 tx=0).stop 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 -- 172.21.15.107:0/3789949611 shutdown_connections 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 --2- 172.21.15.107:0/3789949611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc104250 0x7fcebc10abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 --2- 172.21.15.107:0/3789949611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc103890 0x7fcebc103d10 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 --2- 172.21.15.107:0/3789949611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 0x7fcebc102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.983+0000 7fcec0deb640 1 -- 172.21.15.107:0/3789949611 >> 172.21.15.107:0/3789949611 conn(0x7fcebc0fe020 msgr2=0x7fcebc100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- 172.21.15.107:0/3789949611 shutdown_connections 2024-08-20T06:35:27.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- 172.21.15.107:0/3789949611 wait complete. 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 Processor -- start 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- start start 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 0x7fcebc198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc103890 0x7fcebc1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:27.988 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10e2e0 con 0x7fcebc104250 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10e160 con 0x7fcebc103890 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.984+0000 7fcec0deb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fcebc10e460 con 0x7fcebc102730 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47630/0 (socket says 172.21.15.107:47630) 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 -- 172.21.15.107:0/1933110146 learned_addr learned my addr 172.21.15.107:0/1933110146 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fceba59c640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 0x7fcebc198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fceb9d9b640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc103890 0x7fcebc1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 -- 172.21.15.107:0/1933110146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 msgr2=0x7fcebc198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 0x7fcebc198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 -- 172.21.15.107:0/1933110146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc103890 msgr2=0x7fcebc1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc103890 0x7fcebc1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 -- 172.21.15.107:0/1933110146 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcebc19a5c0 con 0x7fcebc104250 2024-08-20T06:35:27.989 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcebad9d640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fd0 tx=0x7fceb400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4011830 con 0x7fcebc104250 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fceb4016070 con 0x7fcebc104250 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcebc19a8b0 con 0x7fcebc104250 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4010d20 con 0x7fcebc104250 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.985+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcebc1a0f90 con 0x7fcebc104250 2024-08-20T06:35:27.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.986+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcebc109b10 con 0x7fcebc104250 2024-08-20T06:35:27.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.988+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fceb40119d0 con 0x7fcebc104250 2024-08-20T06:35:27.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.988+0000 7fceab7fe640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 0x7fcea40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:27.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.988+0000 7fceba59c640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 0x7fcea40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:27.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.988+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fceb40e3730 con 0x7fcebc104250 2024-08-20T06:35:27.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.989+0000 7fceba59c640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 0x7fcea40b6120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fceac005fd0 tx=0x7fceac009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:27.993 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:27.991+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fceb40aaee0 con 0x7fcebc104250 2024-08-20T06:35:28.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.236+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fcebc02e340 con 0x7fcebc104250 2024-08-20T06:35:28.239 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:28 smithi107 ceph-mon[18495]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:28.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.240+0000 7fceab7fe640 1 -- 172.21.15.107:0/1933110146 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fceb40b31b0 con 0x7fcebc104250 2024-08-20T06:35:28.242 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-08-20T06:35:28.243 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQAwOcRm/OU/DhAA+90+pj9yPXVjDnRJuTHAfg== 2024-08-20T06:35:28.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 msgr2=0x7fcea40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 0x7fcea40b6120 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fceac005fd0 tx=0x7fceac009680 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 msgr2=0x7fcebc199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fd0 tx=0x7fceb400b8e0 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 shutdown_connections 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcebc104250 0x7fcebc199d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fcebc103890 0x7fcebc1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fcea40b3c60 0x7fcea40b6120 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 --2- 172.21.15.107:0/1933110146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc102730 0x7fcebc198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 >> 172.21.15.107:0/1933110146 conn(0x7fcebc0fe020 msgr2=0x7fcebc106ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 shutdown_connections 2024-08-20T06:35:28.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:28.243+0000 7fcec0deb640 1 -- 172.21.15.107:0/1933110146 wait complete. 2024-08-20T06:35:28.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:28 smithi193 ceph-mon[21944]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:28.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:28 smithi183 ceph-mon[22204]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:35:28.917 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:35:28.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-20T06:35:28.917 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-20T06:35:28.955 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph config log 1 --format=json 2024-08-20T06:35:29.242 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:29.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:29 smithi193 ceph-mon[21944]: pgmap v284: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-08-20T06:35:29.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:29 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T06:35:29.305 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:29 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T06:35:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:29 smithi107 ceph-mon[18495]: pgmap v284: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-08-20T06:35:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:29 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T06:35:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:29 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T06:35:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:29 smithi183 ceph-mon[22204]: pgmap v284: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 5.5 KiB/s wr, 9 op/s 2024-08-20T06:35:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:29 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T06:35:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:29 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1933110146' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T06:35:29.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- 172.21.15.107:0/560748302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70102ab0 msgr2=0x7f0f70102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:29.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 --2- 172.21.15.107:0/560748302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70102ab0 0x7f0f70102e90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c009900 tx=0x7f0f6c02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- 172.21.15.107:0/560748302 shutdown_connections 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 --2- 172.21.15.107:0/560748302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f70108e40 0x7f0f7010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 --2- 172.21.15.107:0/560748302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f70108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 --2- 172.21.15.107:0/560748302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70102ab0 0x7f0f70102e90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- 172.21.15.107:0/560748302 >> 172.21.15.107:0/560748302 conn(0x7f0f700fe520 msgr2=0x7f0f70100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- 172.21.15.107:0/560748302 shutdown_connections 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- 172.21.15.107:0/560748302 wait complete. 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 Processor -- start 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.718+0000 7f0f77cab640 1 -- start start 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f70102ab0 0x7f0f70198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f701994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:29.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0f701129b0 con 0x7f0f70108e40 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70112830 con 0x7f0f701084c0 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0f70112b30 con 0x7f0f70102ab0 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47642/0 (socket says 172.21.15.107:47642) 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 -- 172.21.15.107:0/1390329175 learned_addr learned my addr 172.21.15.107:0/1390329175 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f75246640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f701994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 -- 172.21.15.107:0/1390329175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f70102ab0 msgr2=0x7f0f70198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f70102ab0 0x7f0f70198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 -- 172.21.15.107:0/1390329175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 msgr2=0x7f0f701994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f701994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 -- 172.21.15.107:0/1390329175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f7019f010 con 0x7f0f70108e40 2024-08-20T06:35:29.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f75246640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f701994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f76248640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800eaa0 tx=0x7f0f6800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800cdb0 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f7019f300 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0f68005020 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f680116d0 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.719+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f701a5a40 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.720+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f70102e90 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.722+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f0f68004a40 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.722+0000 7f0f66ffd640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 0x7f0f5c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.722+0000 7f0f75a47640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 0x7f0f5c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.723+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f0f680e41f0 con 0x7f0f70108e40 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.723+0000 7f0f75a47640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 0x7f0f5c0b6120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005c20 tx=0x7f0f6c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:29.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.725+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0f68016030 con 0x7f0f70108e40 2024-08-20T06:35:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.904+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f0f7019a090 con 0x7f0f70108e40 2024-08-20T06:35:29.905 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.905+0000 7f0f66ffd640 1 -- 172.21.15.107:0/1390329175 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v50) ==== 86+0+133 (secure 0 0 0) 0x7f0f680ab890 con 0x7f0f70108e40 2024-08-20T06:35:29.906 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 msgr2=0x7f0f5c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 0x7f0f5c0b6120 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c005c20 tx=0x7f0f6c000bc0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 msgr2=0x7f0f7019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800eaa0 tx=0x7f0f6800ef60 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 shutdown_connections 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0f70108e40 0x7f0f7019e840 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0f701084c0 0x7f0f701994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f0f5c0b3c60 0x7f0f5c0b6120 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 --2- 172.21.15.107:0/1390329175 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f70102ab0 0x7f0f70198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:29.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 >> 172.21.15.107:0/1390329175 conn(0x7f0f700fe520 msgr2=0x7f0f70100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 shutdown_connections 2024-08-20T06:35:29.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:29.907+0000 7f0f77cab640 1 -- 172.21.15.107:0/1390329175 wait complete. 2024-08-20T06:35:30.277 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1390329175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T06:35:30.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:30 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1390329175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T06:35:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:30 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1390329175' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T06:35:30.564 INFO:teuthology.orchestra.run.smithi107.stdout:[{"version":50,"timestamp":"2024-08-20T06:34:50.424718+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-20T06:35:30.564 INFO:tasks.ceph_manager:config epoch is 50 2024-08-20T06:35:30.564 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-20T06:35:30.564 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-20T06:35:30.565 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-20T06:35:30.811 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:31.185 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:31 smithi107 ceph-mon[18495]: pgmap v285: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:31.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:31 smithi193 ceph-mon[21944]: pgmap v285: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:31.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4198899509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f81073b0 msgr2=0x7fb3f81077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4198899509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f81073b0 0x7fb3f81077f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009900 tx=0x7fb3ec02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4198899509 shutdown_connections 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4198899509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f8107d30 0x7fb3f810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4198899509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f81073b0 0x7fb3f81077f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4198899509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81019a0 0x7fb3f8101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4198899509 >> 172.21.15.107:0/4198899509 conn(0x7fb3f80fd410 msgr2=0x7fb3f80ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4198899509 shutdown_connections 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.258+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4198899509 wait complete. 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 Processor -- start 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 -- start start 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 0x7fb3f8199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:31.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81073b0 0x7fb3f819a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81118e0 con 0x7fb3f8107d30 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111760 con 0x7fb3f81073b0 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3fe1f0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111a60 con 0x7fb3f81019a0 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3f7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47654/0 (socket says 172.21.15.107:47654) 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4258335651 learned_addr learned my addr 172.21.15.107:0/4258335651 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3f77fe640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 0x7fb3f8199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.259+0000 7fb3f6ffd640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81073b0 0x7fb3f819a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4258335651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 msgr2=0x7fb3f8199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 0x7fb3f8199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4258335651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81073b0 msgr2=0x7fb3f819a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:31.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81073b0 0x7fb3f819a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 -- 172.21.15.107:0/4258335651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f81a51f0 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f77fe640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 0x7fb3f8199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f7fff640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0011830 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81a54e0 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb3f0016070 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0010d20 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.260+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a5a70 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.261+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8101e00 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.262+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fb3f00119d0 con 0x7fb3f8107d30 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.263+0000 7fb3f4ff9640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 0x7fb3e00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.263+0000 7fb3f77fe640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 0x7fb3e00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:31.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.263+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fb3f00e38b0 con 0x7fb3f8107d30 2024-08-20T06:35:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.263+0000 7fb3f77fe640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 0x7fb3e00b6120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fa0 tx=0x7fb3e8009570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:31.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.266+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb3f00ab060 con 0x7fb3f8107d30 2024-08-20T06:35:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:31 smithi183 ceph-mon[22204]: pgmap v285: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:31.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.490+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb3f8109580 con 0x7fb3f8107d30 2024-08-20T06:35:31.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.494+0000 7fb3f4ff9640 1 -- 172.21.15.107:0/4258335651 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+291028 (secure 0 0 0) 0x7fb3f00aa960 con 0x7fb3f8107d30 2024-08-20T06:35:31.499 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 msgr2=0x7fb3e00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 0x7fb3e00b6120 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009fa0 tx=0x7fb3e8009570 comp rx=0 tx=0).stop 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 msgr2=0x7fb3f819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009fd0 tx=0x7fb3f000b8e0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 shutdown_connections 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb3f8107d30 0x7fb3f819a8b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb3f81073b0 0x7fb3f819a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb3e00b3c60 0x7fb3e00b6120 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 --2- 172.21.15.107:0/4258335651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3f81019a0 0x7fb3f8199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:31.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 >> 172.21.15.107:0/4258335651 conn(0x7fb3f80fd410 msgr2=0x7fb3f80ff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:31.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 shutdown_connections 2024-08-20T06:35:31.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:31.497+0000 7fb3fe1f0640 1 -- 172.21.15.107:0/4258335651 wait complete. 2024-08-20T06:35:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:32 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4258335651' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T06:35:32.140 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":1083091367},{"type":"v1","addr":"172.21.15.107:6801","nonce":1083091367}]},"active_addr":"172.21.15.107:6801/1083091367","active_change":"2024-08-20T06:28:01.208713+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24131,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":142658058}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1268797056}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3678314558}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1990235887}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":3399333164}]}]} 2024-08-20T06:35:32.146 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-20T06:35:32.147 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-20T06:35:32.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:35:32.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:32 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4258335651' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T06:35:32.392 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:32 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4258335651' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 -- 172.21.15.107:0/1855068254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107490 msgr2=0x7f73d81078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 --2- 172.21.15.107:0/1855068254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107490 0x7f73d81078f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009900 tx=0x7f73cc02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 -- 172.21.15.107:0/1855068254 shutdown_connections 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 --2- 172.21.15.107:0/1855068254 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8107e30 0x7f73d810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 --2- 172.21.15.107:0/1855068254 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107490 0x7f73d81078f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 --2- 172.21.15.107:0/1855068254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8102630 0x7f73d8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:32.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 -- 172.21.15.107:0/1855068254 >> 172.21.15.107:0/1855068254 conn(0x7f73d8076c80 msgr2=0x7f73d8077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 -- 172.21.15.107:0/1855068254 shutdown_connections 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.860+0000 7f73dded1640 1 -- 172.21.15.107:0/1855068254 wait complete. 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 Processor -- start 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 -- start start 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8102630 0x7f73d8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8107490 0x7f73d81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8111880 con 0x7f73d8107e30 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8111700 con 0x7f73d8102630 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73dded1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8111a00 con 0x7f73d8107490 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73d7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:32.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73d7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47674/0 (socket says 172.21.15.107:47674) 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73d7fff640 1 -- 172.21.15.107:0/2103290857 learned_addr learned my addr 172.21.15.107:0/2103290857 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73d6ffd640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8107490 0x7f73d81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.861+0000 7f73d77fe640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8102630 0x7f73d8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 -- 172.21.15.107:0/2103290857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8107490 msgr2=0x7f73d81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8107490 0x7f73d81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 -- 172.21.15.107:0/2103290857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8102630 msgr2=0x7f73d8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8102630 0x7f73d8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 -- 172.21.15.107:0/2103290857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d819df20 con 0x7f73d8107e30 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d7fff640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d750 tx=0x7f73d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0017070 con 0x7f73d8107e30 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d819e210 con 0x7f73d8107e30 2024-08-20T06:35:32.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f73d00054b0 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004b90 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.862+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d81a48d0 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.863+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d8102a10 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.864+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f73d0004d30 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.865+0000 7f73d4ff9640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 0x7f73c00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.865+0000 7f73d77fe640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 0x7f73c00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.865+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f73d00e35d0 con 0x7f73d8107e30 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.866+0000 7f73d77fe640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 0x7f73c00b6120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c800a400 tx=0x7f73c800d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:32.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:32.869+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f73d00aa620 con 0x7f73d8107e30 2024-08-20T06:35:33.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.044+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73d81095c0 con 0x7f73d8107e30 2024-08-20T06:35:33.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.046+0000 7f73d4ff9640 1 -- 172.21.15.107:0/2103290857 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+22371 (secure 0 0 0) 0x7f73d00aeb70 con 0x7f73d8107e30 2024-08-20T06:35:33.047 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:33.047 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":71,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:35:00.478609+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 msgr2=0x7f73c00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 0x7f73c00b6120 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73c800a400 tx=0x7f73c800d040 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 msgr2=0x7f73d819d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f73d000d750 tx=0x7f73d000dc10 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 shutdown_connections 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f73d8107e30 0x7f73d819d750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f73c00b3c60 0x7f73c00b6120 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73d8107490 0x7f73d81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 --2- 172.21.15.107:0/2103290857 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f73d8102630 0x7f73d8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 >> 172.21.15.107:0/2103290857 conn(0x7f73d8076c80 msgr2=0x7f73d8100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 shutdown_connections 2024-08-20T06:35:33.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:33.048+0000 7f73dded1640 1 -- 172.21.15.107:0/2103290857 wait complete. 2024-08-20T06:35:33.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:33 smithi193 ceph-mon[21944]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:33.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:33 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2103290857' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:33 smithi107 ceph-mon[18495]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:33 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2103290857' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:33 smithi183 ceph-mon[22204]: pgmap v286: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 3.1 KiB/s wr, 7 op/s 2024-08-20T06:35:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:33 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2103290857' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:33.698 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-20T06:35:33.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:35:33.946 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:34.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- 172.21.15.107:0/633906356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c4108e40 msgr2=0x7f62c410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- 172.21.15.107:0/633906356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c4108e40 0x7f62c410f640 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009ca0 tx=0x7f62bc02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- 172.21.15.107:0/633906356 shutdown_connections 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- 172.21.15.107:0/633906356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c4108e40 0x7f62c410f640 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- 172.21.15.107:0/633906356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c41084c0 0x7f62c4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- 172.21.15.107:0/633906356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 0x7f62c4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- 172.21.15.107:0/633906356 >> 172.21.15.107:0/633906356 conn(0x7f62c40fe500 msgr2=0x7f62c4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- 172.21.15.107:0/633906356 shutdown_connections 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- 172.21.15.107:0/633906356 wait complete. 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 Processor -- start 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- start start 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 0x7f62c4198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:34.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c4108e40 0x7f62c419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4112a00 con 0x7f62c41084c0 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4112880 con 0x7f62c4102ab0 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62c9c56640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f62c4112b80 con 0x7f62c4108e40 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.386+0000 7f62bbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47688/0 (socket says 172.21.15.107:47688) 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 -- 172.21.15.107:0/1741193097 learned_addr learned my addr 172.21.15.107:0/1741193097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c37fe640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 0x7f62c4198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c3fff640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c4108e40 0x7f62c419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 -- 172.21.15.107:0/1741193097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c4108e40 msgr2=0x7f62c419e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c4108e40 0x7f62c419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 -- 172.21.15.107:0/1741193097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 msgr2=0x7f62c4198f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 0x7f62c4198f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 -- 172.21.15.107:0/1741193097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c419efc0 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62bbfff640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b4a0 tx=0x7f62ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac014070 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f62ac010aa0 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c419f2b0 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac017b70 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.387+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c41a5970 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.388+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c4102e90 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.390+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f62ac017070 con 0x7f62c41084c0 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.390+0000 7f62c17fa640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 0x7f62b00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:34.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.390+0000 7f62c37fe640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 0x7f62b00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:34.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.390+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f62ac0e3260 con 0x7f62c41084c0 2024-08-20T06:35:34.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.391+0000 7f62c37fe640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 0x7f62b00b6120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62b400bbe0 tx=0x7f62b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:34.427 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.393+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f62ac0aaa10 con 0x7f62c41084c0 2024-08-20T06:35:34.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.573+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62c410a690 con 0x7f62c41084c0 2024-08-20T06:35:34.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.574+0000 7f62c17fa640 1 -- 172.21.15.107:0/1741193097 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) ==== 74+0+22371 (secure 0 0 0) 0x7f62ac0aa310 con 0x7f62c41084c0 2024-08-20T06:35:34.575 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:35:34.575 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":71,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:35:00.478609+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 msgr2=0x7f62b00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 0x7f62b00b6120 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62b400bbe0 tx=0x7f62b400b390 comp rx=0 tx=0).stop 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 msgr2=0x7f62c4199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b4a0 tx=0x7f62ac00b960 comp rx=0 tx=0).stop 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 shutdown_connections 2024-08-20T06:35:34.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f62b00b3c60 0x7f62b00b6120 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62c4108e40 0x7f62c419e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f62c41084c0 0x7f62c4199450 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 --2- 172.21.15.107:0/1741193097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f62c4102ab0 0x7f62c4198f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 >> 172.21.15.107:0/1741193097 conn(0x7f62c40fe500 msgr2=0x7f62c41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 shutdown_connections 2024-08-20T06:35:34.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:34.576+0000 7f62c9c56640 1 -- 172.21.15.107:0/1741193097 wait complete. 2024-08-20T06:35:35.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:35 smithi107 ceph-mon[18495]: pgmap v287: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-08-20T06:35:35.228 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:35 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1741193097' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:35.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-20T06:35:35.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-20T06:35:35.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-20T06:35:35.260 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-20T06:35:35.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-20T06:35:35.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-20T06:35:35.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-20T06:35:35.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-20T06:35:35.262 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-20T06:35:35.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-20T06:35:35.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-20T06:35:35.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-20T06:35:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:35 smithi193 ceph-mon[21944]: pgmap v287: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-08-20T06:35:35.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1741193097' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:35 smithi183 ceph-mon[22204]: pgmap v287: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2024-08-20T06:35:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:35 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1741193097' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:35:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.976 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.986 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.989 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.990 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:35.990 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:36.488 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:36.489 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:36.489 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 -- 172.21.15.107:0/805228401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934102ad0 msgr2=0x7f1934102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 --2- 172.21.15.107:0/805228401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934102ad0 0x7f1934102eb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1924009900 tx=0x7f192402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 -- 172.21.15.107:0/805228401 shutdown_connections 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 --2- 172.21.15.107:0/805228401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1934108e60 0x7f193410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 --2- 172.21.15.107:0/805228401 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f19341084e0 0x7f1934108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 --2- 172.21.15.107:0/805228401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934102ad0 0x7f1934102eb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 -- 172.21.15.107:0/805228401 >> 172.21.15.107:0/805228401 conn(0x7f19340fe540 msgr2=0x7f1934100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 -- 172.21.15.107:0/805228401 shutdown_connections 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.845+0000 7f193a7eb640 1 -- 172.21.15.107:0/805228401 wait complete. 2024-08-20T06:35:36.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 Processor -- start 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 -- start start 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1934102ad0 0x7f1934198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19341084e0 0x7f19341993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 0x7f193419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f19341129d0 con 0x7f1934108e60 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112850 con 0x7f1934102ad0 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f193a7eb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1934112b50 con 0x7f19341084e0 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 0x7f193419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 0x7f193419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40640/0 (socket says 172.21.15.107:40640) 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 -- 172.21.15.107:0/1337004207 learned_addr learned my addr 172.21.15.107:0/1337004207 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f19337fe640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19341084e0 0x7f19341993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:36.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1933fff640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1934102ad0 0x7f1934198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 -- 172.21.15.107:0/1337004207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19341084e0 msgr2=0x7f19341993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19341084e0 0x7f19341993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 -- 172.21.15.107:0/1337004207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1934102ad0 msgr2=0x7f1934198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1934102ad0 0x7f1934198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1938d88640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f193419ef50 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.846+0000 7f1933fff640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1934102ad0 0x7f1934198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f1938d88640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 0x7f193419e780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f192c00eb10 tx=0x7f192c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c004b20 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f192c004cc0 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f193419f240 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c010690 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.847+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19341a5900 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.848+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1934063610 con 0x7f1934108e60 2024-08-20T06:35:36.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.849+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f192c005000 con 0x7f1934108e60 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19317fa640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f191c0b3c60 0x7f191c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f1933fff640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f191c0b3c60 0x7f191c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f192c0e45e0 con 0x7f1934108e60 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19317fa640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] conn(0x7f191c0bd4b0 0x7f191c0bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f192c015e80 con 0x7f191c0bd4b0 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f192c0e4ac0 con 0x7f1934108e60 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19337fe640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] conn(0x7f191c0bd4b0 0x7f191c0bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f1933fff640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f191c0b3c60 0x7f191c0b6120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1924000c00 tx=0x7f1924032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.850+0000 7f19337fe640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] conn(0x7f191c0bd4b0 0x7f191c0bf8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:36.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.851+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== osd.0 v2:172.21.15.107:6802/3184473884 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f192c015e80 con 0x7f191c0bd4b0 2024-08-20T06:35:36.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 --> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1934063a90 con 0x7f191c0bd4b0 2024-08-20T06:35:36.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f19317fa640 1 -- 172.21.15.107:0/1337004207 <== osd.0 v2:172.21.15.107:6802/3184473884 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1934063a90 con 0x7f191c0bd4b0 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] conn(0x7f191c0bd4b0 msgr2=0x7f191c0bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] conn(0x7f191c0bd4b0 0x7f191c0bf8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f191c0b3c60 msgr2=0x7f191c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f191c0b3c60 0x7f191c0b6120 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1924000c00 tx=0x7f1924032c40 comp rx=0 tx=0).stop 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 msgr2=0x7f193419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f193a7eb640 1 --2- 172.21.15.107:0/1337004207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1934108e60 0x7f193419e780 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f192c00eb10 tx=0x7f192c00c2d0 comp rx=0 tx=0).stop 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.865+0000 7f1938d88640 1 -- 172.21.15.107:0/1337004207 reap_dead start 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.866+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 shutdown_connections 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.866+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 >> 172.21.15.107:0/1337004207 conn(0x7f19340fe540 msgr2=0x7f1934100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.866+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 shutdown_connections 2024-08-20T06:35:36.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.866+0000 7f193a7eb640 1 -- 172.21.15.107:0/1337004207 wait complete. 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 -- 172.21.15.107:0/4001329458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 msgr2=0x7f2b74102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4001329458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74102eb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009900 tx=0x7f2b6402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 -- 172.21.15.107:0/4001329458 shutdown_connections 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4001329458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b74108e60 0x7f2b7410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4001329458 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 0x7f2b74108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4001329458 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74102eb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 -- 172.21.15.107:0/4001329458 >> 172.21.15.107:0/4001329458 conn(0x7f2b740fe540 msgr2=0x7f2b74100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 -- 172.21.15.107:0/4001329458 shutdown_connections 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.740+0000 7f2b79f26640 1 -- 172.21.15.107:0/4001329458 wait complete. 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 Processor -- start 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 -- start start 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 0x7f2b741993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b74108e60 0x7f2b7419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741129d0 con 0x7f2b74102ad0 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74112850 con 0x7f2b741084e0 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b79f26640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b74112b50 con 0x7f2b74108e60 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:37.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74198ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40620/0 (socket says 172.21.15.107:40620) 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 -- 172.21.15.107:0/4194439697 learned_addr learned my addr 172.21.15.107:0/4194439697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b73fff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b74108e60 0x7f2b7419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 0x7f2b741993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 -- 172.21.15.107:0/4194439697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b74108e60 msgr2=0x7f2b7419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b74108e60 0x7f2b7419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 -- 172.21.15.107:0/4194439697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 msgr2=0x7f2b741993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 0x7f2b741993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7419ef50 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b741084e0 0x7f2b741993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.741+0000 7f2b737fe640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74198ea0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009230 tx=0x7f2b64004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.742+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6403f030 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.742+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7419f1e0 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.742+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b64003a80 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.742+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64035070 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.742+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b741a5900 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.743+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2b740635d0 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.744+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2b64042030 con 0x7f2b74102ad0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b717fa640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b600b3c60 0x7f2b600b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b600b3c60 0x7f2b600b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:37.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f2b64104850 con 0x7f2b74102ad0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b717fa640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] conn(0x7f2b600bd4b0 0x7f2b600bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2b600bff80 con 0x7f2b600bd4b0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f2b64104d30 con 0x7f2b74102ad0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b73fff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] conn(0x7f2b600bd4b0 0x7f2b600bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b6bfff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b600b3c60 0x7f2b600b6120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005d00 tx=0x7f2b5c005ed0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.745+0000 7f2b73fff640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] conn(0x7f2b600bd4b0 0x7f2b600bf8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.746+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== osd.6 v2:172.21.15.107:6818/2830948691 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2b600bff80 con 0x7f2b600bd4b0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.759+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 --> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2b74063a50 con 0x7f2b600bd4b0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b717fa640 1 -- 172.21.15.107:0/4194439697 <== osd.6 v2:172.21.15.107:6818/2830948691 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2b74063a50 con 0x7f2b600bd4b0 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] conn(0x7f2b600bd4b0 msgr2=0x7f2b600bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] conn(0x7f2b600bd4b0 0x7f2b600bf8d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b600b3c60 msgr2=0x7f2b600b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b600b3c60 0x7f2b600b6120 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c005d00 tx=0x7f2b5c005ed0 comp rx=0 tx=0).stop 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 msgr2=0x7f2b74198ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 --2- 172.21.15.107:0/4194439697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b74102ad0 0x7f2b74198ea0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b64009230 tx=0x7f2b64004270 comp rx=0 tx=0).stop 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b73fff640 1 -- 172.21.15.107:0/4194439697 reap_dead start 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 shutdown_connections 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 >> 172.21.15.107:0/4194439697 conn(0x7f2b740fe540 msgr2=0x7f2b74100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 shutdown_connections 2024-08-20T06:35:37.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:36.760+0000 7f2b79f26640 1 -- 172.21.15.107:0/4194439697 wait complete. 2024-08-20T06:35:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:37 smithi107 ceph-mon[18495]: pgmap v288: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-20T06:35:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:37 smithi183 ceph-mon[22204]: pgmap v288: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-20T06:35:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:37 smithi193 ceph-mon[21944]: pgmap v288: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-08-20T06:35:37.902 INFO:teuthology.orchestra.run.smithi107.stdout:167503724587 2024-08-20T06:35:37.903 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-20T06:35:38.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.734+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2945263488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981039a0 msgr2=0x7fdb98103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.734+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2945263488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981039a0 0x7fdb98103d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdb88009900 tx=0x7fdb8802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.734+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2945263488 shutdown_connections 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.734+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2945263488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb9810ce00 0x7fdb9810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2945263488 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb981042c0 0x7fdb9810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2945263488 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981039a0 0x7fdb98103d80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2945263488 >> 172.21.15.107:0/2945263488 conn(0x7fdb980fd400 msgr2=0x7fdb980ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2945263488 shutdown_connections 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2945263488 wait complete. 2024-08-20T06:35:38.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 Processor -- start 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.735+0000 7fdb9d0f5640 1 -- start start 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb981039a0 0x7fdb98198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 0x7fdb981993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9810ce00 0x7fdb9819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98112920 con 0x7fdb981042c0 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981127a0 con 0x7fdb9810ce00 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9d0f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98112aa0 con 0x7fdb981039a0 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 0x7fdb981993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 0x7fdb981993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40648/0 (socket says 172.21.15.107:40648) 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 -- 172.21.15.107:0/2670297471 learned_addr learned my addr 172.21.15.107:0/2670297471 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb96d9d640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb981039a0 0x7fdb98198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9759e640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9810ce00 0x7fdb9819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:38.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 -- 172.21.15.107:0/2670297471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb981039a0 msgr2=0x7fdb98198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb981039a0 0x7fdb98198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 -- 172.21.15.107:0/2670297471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9810ce00 msgr2=0x7fdb9819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdb9810ce00 0x7fdb9819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9819ef40 con 0x7fdb981042c0 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb9659c640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 0x7fdb981993d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00c970 tx=0x7fdb8c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.736+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c004220 con 0x7fdb981042c0 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.737+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9819f230 con 0x7fdb981042c0 2024-08-20T06:35:38.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.737+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdb8c007e40 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.737+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c003d20 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.737+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb981a5ac0 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.738+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb98063610 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fdb8c0079b0 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb800b3c60 0x7fdb800b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb96d9d640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb800b3c60 0x7fdb800b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fdb8c0e3cb0 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] conn(0x7fdb800bd4b0 0x7fdb800bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb8c016ec0 con 0x7fdb800bd4b0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.740+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fdb8c0e4060 con 0x7fdb981042c0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.741+0000 7fdb9759e640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] conn(0x7fdb800bd4b0 0x7fdb800bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.741+0000 7fdb96d9d640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb800b3c60 0x7fdb800b6120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb880057c0 tx=0x7fdb880058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:38.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.741+0000 7fdb9759e640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] conn(0x7fdb800bd4b0 0x7fdb800bf8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.742+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== osd.3 v2:172.21.15.107:6810/3836170124 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb8c016ec0 con 0x7fdb800bd4b0 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.758+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 --> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb98063a10 con 0x7fdb800bd4b0 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.758+0000 7fdb7ffff640 1 -- 172.21.15.107:0/2670297471 <== osd.3 v2:172.21.15.107:6810/3836170124 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdb98063a10 con 0x7fdb800bd4b0 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] conn(0x7fdb800bd4b0 msgr2=0x7fdb800bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] conn(0x7fdb800bd4b0 0x7fdb800bf8d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb800b3c60 msgr2=0x7fdb800b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdb800b3c60 0x7fdb800b6120 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdb880057c0 tx=0x7fdb880058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 msgr2=0x7fdb981993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 --2- 172.21.15.107:0/2670297471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdb981042c0 0x7fdb981993d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00c970 tx=0x7fdb8c00ce30 comp rx=0 tx=0).stop 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9759e640 1 -- 172.21.15.107:0/2670297471 reap_dead start 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 shutdown_connections 2024-08-20T06:35:38.882 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 >> 172.21.15.107:0/2670297471 conn(0x7fdb980fd400 msgr2=0x7fdb980fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 shutdown_connections 2024-08-20T06:35:38.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:38.759+0000 7fdb9d0f5640 1 -- 172.21.15.107:0/2670297471 wait complete. 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.276+0000 7feafcdc8640 1 -- 172.21.15.107:0/161036827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81039a0 msgr2=0x7feaf8103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.276+0000 7feafcdc8640 1 --2- 172.21.15.107:0/161036827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81039a0 0x7feaf8103d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feaec009900 tx=0x7feaec02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 -- 172.21.15.107:0/161036827 shutdown_connections 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 --2- 172.21.15.107:0/161036827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 0x7feaf810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 --2- 172.21.15.107:0/161036827 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaf81042c0 0x7feaf810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.280 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 --2- 172.21.15.107:0/161036827 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81039a0 0x7feaf8103d80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 -- 172.21.15.107:0/161036827 >> 172.21.15.107:0/161036827 conn(0x7feaf80fd400 msgr2=0x7feaf80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 -- 172.21.15.107:0/161036827 shutdown_connections 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 -- 172.21.15.107:0/161036827 wait complete. 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 Processor -- start 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.277+0000 7feafcdc8640 1 -- start start 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaf81039a0 0x7feaf8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 0x7feaf8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 0x7feaf819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112920 con 0x7feaf81042c0 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7feaf81127a0 con 0x7feaf81039a0 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feafcdc8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7feaf8112aa0 con 0x7feaf810ce00 2024-08-20T06:35:39.281 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 0x7feaf8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 0x7feaf8199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40676/0 (socket says 172.21.15.107:40676) 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 -- 172.21.15.107:0/3758424793 learned_addr learned my addr 172.21.15.107:0/3758424793 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf659c640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaf81039a0 0x7feaf8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf6d9d640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 0x7feaf819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 -- 172.21.15.107:0/3758424793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 msgr2=0x7feaf819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 0x7feaf819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 -- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaf81039a0 msgr2=0x7feaf8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7feaf81039a0 0x7feaf8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf819eec0 con 0x7feaf81042c0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf6d9d640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaf810ce00 0x7feaf819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.278+0000 7feaf5d9b640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 0x7feaf8199350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feae800c970 tx=0x7feae800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8007bb0 con 0x7feaf81042c0 2024-08-20T06:35:39.282 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf819f1b0 con 0x7feaf81042c0 2024-08-20T06:35:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf81a5a40 con 0x7feaf81042c0 2024-08-20T06:35:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7feae8007d50 con 0x7feaf81042c0 2024-08-20T06:35:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae800f6b0 con 0x7feaf81042c0 2024-08-20T06:35:39.283 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.279+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fead8001060 con 0x7feaf81042c0 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.281+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7feae800f850 con 0x7feaf81042c0 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feae77fe640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feae00b3c60 0x7feae00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feaf659c640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feae00b3c60 0x7feae00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7feae80e42f0 con 0x7feaf81042c0 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feae77fe640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] conn(0x7feae00bd4b0 0x7feae00bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feae8013e80 con 0x7feae00bd4b0 2024-08-20T06:35:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7feae80e47d0 con 0x7feaf81042c0 2024-08-20T06:35:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.282+0000 7feaf659c640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feae00b3c60 0x7feae00b6120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feaec0057c0 tx=0x7feaec0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.283+0000 7feaf6d9d640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] conn(0x7feae00bd4b0 0x7feae00bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.283+0000 7feaf6d9d640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] conn(0x7feae00bd4b0 0x7feae00bf8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.285 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.284+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== osd.10 v2:172.21.15.183:6824/2667647347 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feae8013e80 con 0x7feae00bd4b0 2024-08-20T06:35:39.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.297+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 --> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fead8002e00 con 0x7feae00bd4b0 2024-08-20T06:35:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feae77fe640 1 -- 172.21.15.107:0/3758424793 <== osd.10 v2:172.21.15.183:6824/2667647347 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fead8002e00 con 0x7feae00bd4b0 2024-08-20T06:35:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] conn(0x7feae00bd4b0 msgr2=0x7feae00bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] conn(0x7feae00bd4b0 0x7feae00bf8d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feae00b3c60 msgr2=0x7feae00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7feae00b3c60 0x7feae00b6120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7feaec0057c0 tx=0x7feaec0058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 msgr2=0x7feaf8199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 --2- 172.21.15.107:0/3758424793 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feaf81042c0 0x7feaf8199350 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feae800c970 tx=0x7feae800ce30 comp rx=0 tx=0).stop 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feaf6d9d640 1 -- 172.21.15.107:0/3758424793 reap_dead start 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 shutdown_connections 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.298+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 >> 172.21.15.107:0/3758424793 conn(0x7feaf80fd400 msgr2=0x7feaf80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.299+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 shutdown_connections 2024-08-20T06:35:39.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.299+0000 7feafcdc8640 1 -- 172.21.15.107:0/3758424793 wait complete. 2024-08-20T06:35:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:39 smithi107 ceph-mon[18495]: pgmap v289: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-20T06:35:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:39 smithi183 ceph-mon[22204]: pgmap v289: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-20T06:35:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:39 smithi193 ceph-mon[21944]: pgmap v289: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.922+0000 7efdc9823640 1 -- 172.21.15.107:0/1171730947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 msgr2=0x7efdc410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.922+0000 7efdc9823640 1 --2- 172.21.15.107:0/1171730947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc410ca20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009900 tx=0x7efdb802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 -- 172.21.15.107:0/1171730947 shutdown_connections 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 --2- 172.21.15.107:0/1171730947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdc410cf60 0x7efdc410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 --2- 172.21.15.107:0/1171730947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc410ca20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 --2- 172.21.15.107:0/1171730947 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc4101520 0x7efdc4101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 -- 172.21.15.107:0/1171730947 >> 172.21.15.107:0/1171730947 conn(0x7efdc4076cc0 msgr2=0x7efdc40770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 -- 172.21.15.107:0/1171730947 shutdown_connections 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 -- 172.21.15.107:0/1171730947 wait complete. 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.923+0000 7efdc9823640 1 Processor -- start 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 -- start start 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdc4101520 0x7efdc4198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc410cf60 0x7efdc419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4112950 con 0x7efdc410a630 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efdc41127d0 con 0x7efdc410cf60 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc9823640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efdc4112ad0 con 0x7efdc4101520 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40680/0 (socket says 172.21.15.107:40680) 2024-08-20T06:35:39.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 -- 172.21.15.107:0/2201671456 learned_addr learned my addr 172.21.15.107:0/2201671456 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc2ffd640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdc4101520 0x7efdc4198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc37fe640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc410cf60 0x7efdc419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 -- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdc4101520 msgr2=0x7efdc4198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdc4101520 0x7efdc4198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 -- 172.21.15.107:0/2201671456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc410cf60 msgr2=0x7efdc419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efdc410cf60 0x7efdc419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.924+0000 7efdc27fc640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc419ef50 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc27fc640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc41993e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb803f030 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdc419f1e0 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efdb8003a80 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb8035070 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.925+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdc41a5900 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.926+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efdc4100d10 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.927+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7efdb8042030 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.928+0000 7efdc8821640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efdac0b3c60 0x7efdac0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.928+0000 7efdc2ffd640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efdac0b3c60 0x7efdac0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.928+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7efdb8104750 con 0x7efdc410a630 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.928+0000 7efdc8821640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] conn(0x7efdac0bd4b0 0x7efdac0bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:39.957 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.928+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efdb80038d0 con 0x7efdac0bd4b0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.929+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7efdb8104bb0 con 0x7efdc410a630 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.929+0000 7efdc37fe640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] conn(0x7efdac0bd4b0 0x7efdac0bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.929+0000 7efdc2ffd640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efdac0b3c60 0x7efdac0b6120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005fd0 tx=0x7efdb4009680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.929+0000 7efdc37fe640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] conn(0x7efdac0bd4b0 0x7efdac0bf8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.930+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== osd.11 v2:172.21.15.193:6824/735254759 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efdb80038d0 con 0x7efdac0bd4b0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.945+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 --> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efdc406ab30 con 0x7efdac0bd4b0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc8821640 1 -- 172.21.15.107:0/2201671456 <== osd.11 v2:172.21.15.193:6824/735254759 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efdc406ab30 con 0x7efdac0bd4b0 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] conn(0x7efdac0bd4b0 msgr2=0x7efdac0bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] conn(0x7efdac0bd4b0 0x7efdac0bf8d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efdac0b3c60 msgr2=0x7efdac0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efdac0b3c60 0x7efdac0b6120 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efdb4005fd0 tx=0x7efdb4009680 comp rx=0 tx=0).stop 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 msgr2=0x7efdc41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:39.958 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 --2- 172.21.15.107:0/2201671456 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efdc410a630 0x7efdc41993e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009230 tx=0x7efdb8004270 comp rx=0 tx=0).stop 2024-08-20T06:35:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc37fe640 1 -- 172.21.15.107:0/2201671456 reap_dead start 2024-08-20T06:35:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 shutdown_connections 2024-08-20T06:35:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 >> 172.21.15.107:0/2201671456 conn(0x7efdc4076cc0 msgr2=0x7efdc41002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 shutdown_connections 2024-08-20T06:35:39.959 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:39.946+0000 7efdc9823640 1 -- 172.21.15.107:0/2201671456 wait complete. 2024-08-20T06:35:40.689 INFO:teuthology.orchestra.run.smithi107.stdout:98784247866 2024-08-20T06:35:40.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-20T06:35:41.090 INFO:teuthology.orchestra.run.smithi107.stdout:34359738440 2024-08-20T06:35:41.091 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-20T06:35:41.348 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:41 smithi107 ceph-mon[18495]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:41 smithi183 ceph-mon[22204]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:41 smithi193 ceph-mon[21944]: pgmap v290: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:42.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 -- 172.21.15.107:0/2461785653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47881084c0 msgr2=0x7f4788108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 --2- 172.21.15.107:0/2461785653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47881084c0 0x7f4788108900 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f477c009900 tx=0x7f477c02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 -- 172.21.15.107:0/2461785653 shutdown_connections 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 --2- 172.21.15.107:0/2461785653 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 0x7f478810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 --2- 172.21.15.107:0/2461785653 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f47881084c0 0x7f4788108900 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 --2- 172.21.15.107:0/2461785653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4788102ab0 0x7f4788102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 -- 172.21.15.107:0/2461785653 >> 172.21.15.107:0/2461785653 conn(0x7f47880fe500 msgr2=0x7f4788100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 -- 172.21.15.107:0/2461785653 shutdown_connections 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.137+0000 7f478db07640 1 -- 172.21.15.107:0/2461785653 wait complete. 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 Processor -- start 2024-08-20T06:35:42.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 -- start start 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 0x7f47880751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47881084c0 0x7f4788075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 0x7f4788075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f47881129b0 con 0x7f4788102ab0 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4788112830 con 0x7f4788108e40 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f478db07640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4788112b30 con 0x7f47881084c0 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 0x7f47880751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 0x7f47880751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40710/0 (socket says 172.21.15.107:40710) 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 -- 172.21.15.107:0/4148295791 learned_addr learned my addr 172.21.15.107:0/4148295791 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f4787fff640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 0x7f4788075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:42.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f4786ffd640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47881084c0 0x7f4788075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 -- 172.21.15.107:0/4148295791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47881084c0 msgr2=0x7f4788075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47881084c0 0x7f4788075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 -- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 msgr2=0x7f4788075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 0x7f4788075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f47877fe640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47881a72e0 con 0x7f4788102ab0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.138+0000 7f4787fff640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4788108e40 0x7f4788075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f47877fe640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 0x7f47880751d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f477800d770 tx=0x7f477800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778017070 con 0x7f4788102ab0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f47780054b0 con 0x7f4788102ab0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47881a75d0 con 0x7f4788102ab0 2024-08-20T06:35:42.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4778004c60 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.139+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47881a7ae0 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.140+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4788063610 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.141+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f4778004e00 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4784ff9640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f47700b3ce0 0x7f47700b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4786ffd640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f47700b3ce0 0x7f47700b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f47780e3aa0 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4784ff9640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] conn(0x7f47700bd530 0x7f47700bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f47700c0000 con 0x7f47700bd530 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f47780e3f80 con 0x7f4788102ab0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.142+0000 7f4786ffd640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f47700b3ce0 0x7f47700b61a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:42.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.143+0000 7f4787fff640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] conn(0x7f47700bd530 0x7f47700bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.143+0000 7f4787fff640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] conn(0x7f47700bd530 0x7f47700bf950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.144+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== osd.1 v2:172.21.15.183:6800/2162362081 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f47700c0000 con 0x7f47700bd530 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.157+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 --> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4788063a90 con 0x7f47700bd530 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f4784ff9640 1 -- 172.21.15.107:0/4148295791 <== osd.1 v2:172.21.15.183:6800/2162362081 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4788063a90 con 0x7f47700bd530 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] conn(0x7f47700bd530 msgr2=0x7f47700bf950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] conn(0x7f47700bd530 0x7f47700bf950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f47700b3ce0 msgr2=0x7f47700b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f47700b3ce0 0x7f47700b61a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f477c000c00 tx=0x7f477c005a20 comp rx=0 tx=0).stop 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 msgr2=0x7f47880751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 --2- 172.21.15.107:0/4148295791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4788102ab0 0x7f47880751d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f477800d770 tx=0x7f477800dc30 comp rx=0 tx=0).stop 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f4787fff640 1 -- 172.21.15.107:0/4148295791 reap_dead start 2024-08-20T06:35:42.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 shutdown_connections 2024-08-20T06:35:42.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 >> 172.21.15.107:0/4148295791 conn(0x7f47880fe500 msgr2=0x7f47881008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:42.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 shutdown_connections 2024-08-20T06:35:42.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.158+0000 7f478db07640 1 -- 172.21.15.107:0/4148295791 wait complete. 2024-08-20T06:35:42.376 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:42.850 INFO:teuthology.orchestra.run.smithi107.stdout:51539607619 2024-08-20T06:35:42.850 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-20T06:35:43.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:43 smithi107 ceph-mon[18495]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:43.247 INFO:teuthology.orchestra.run.smithi107.stdout:257698037785 2024-08-20T06:35:43.247 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-20T06:35:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:43 smithi183 ceph-mon[22204]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:43 smithi193 ceph-mon[21944]: pgmap v291: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 597 B/s wr, 6 op/s 2024-08-20T06:35:43.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 -- 172.21.15.107:0/1323658359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 msgr2=0x7f416810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1323658359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416810e520 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4160009ca0 tx=0x7f416002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 -- 172.21.15.107:0/1323658359 shutdown_connections 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1323658359 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416810e520 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1323658359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41681073a0 0x7f41681077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1323658359 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4168101990 0x7f4168101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 -- 172.21.15.107:0/1323658359 >> 172.21.15.107:0/1323658359 conn(0x7f41680fd400 msgr2=0x7f41680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 -- 172.21.15.107:0/1323658359 shutdown_connections 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.779+0000 7f416fbc2640 1 -- 172.21.15.107:0/1323658359 wait complete. 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 Processor -- start 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 -- start start 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4168101990 0x7f4168198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41681073a0 0x7f41681993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f41681129e0 con 0x7f4168107d20 2024-08-20T06:35:43.812 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4168112860 con 0x7f4168101990 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416fbc2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4168112b60 con 0x7f41681073a0 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416e15f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416e15f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40718/0 (socket says 172.21.15.107:40718) 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.780+0000 7f416e15f640 1 -- 172.21.15.107:0/1061424140 learned_addr learned my addr 172.21.15.107:0/1061424140 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416d95e640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4168101990 0x7f4168198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416d15d640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41681073a0 0x7f41681993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 -- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41681073a0 msgr2=0x7f41681993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41681073a0 0x7f41681993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 -- 172.21.15.107:0/1061424140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4168101990 msgr2=0x7f4168198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4168101990 0x7f4168198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416819ef60 con 0x7f4168107d20 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416e15f640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416819e790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4160005cc0 tx=0x7f4160032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160032e00 con 0x7f4168107d20 2024-08-20T06:35:43.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416819f1f0 con 0x7f4168107d20 2024-08-20T06:35:43.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f41600044d0 con 0x7f4168107d20 2024-08-20T06:35:43.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160030b70 con 0x7f4168107d20 2024-08-20T06:35:43.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.781+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41681a5910 con 0x7f4168107d20 2024-08-20T06:35:43.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.782+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4168063610 con 0x7f4168107d20 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.783+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f4160004680 con 0x7f4168107d20 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f415effd640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f41540b3c60 0x7f41540b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f416d95e640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f41540b3c60 0x7f41540b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f41601051e0 con 0x7f4168107d20 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f415effd640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] conn(0x7f41540bd4b0 0x7f41540bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4160047ec0 con 0x7f41540bd4b0 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f41601056c0 con 0x7f4168107d20 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.784+0000 7f416d95e640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f41540b3c60 0x7f41540b6120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f416400bbe0 tx=0x7f416400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.785+0000 7f416d15d640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] conn(0x7f41540bd4b0 0x7f41540bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:43.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.785+0000 7f416d15d640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] conn(0x7f41540bd4b0 0x7f41540bf8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.786+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== osd.5 v2:172.21.15.193:6808/2859012994 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4160047ec0 con 0x7f41540bd4b0 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.799+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 --> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4168063a10 con 0x7f41540bd4b0 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.800+0000 7f415effd640 1 -- 172.21.15.107:0/1061424140 <== osd.5 v2:172.21.15.193:6808/2859012994 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4168063a10 con 0x7f41540bd4b0 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] conn(0x7f41540bd4b0 msgr2=0x7f41540bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] conn(0x7f41540bd4b0 0x7f41540bf8d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f41540b3c60 msgr2=0x7f41540b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f41540b3c60 0x7f41540b6120 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f416400bbe0 tx=0x7f416400b390 comp rx=0 tx=0).stop 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 msgr2=0x7f416819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 --2- 172.21.15.107:0/1061424140 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4168107d20 0x7f416819e790 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4160005cc0 tx=0x7f4160032a10 comp rx=0 tx=0).stop 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416e15f640 1 -- 172.21.15.107:0/1061424140 reap_dead start 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 shutdown_connections 2024-08-20T06:35:43.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 >> 172.21.15.107:0/1061424140 conn(0x7f41680fd400 msgr2=0x7f41680ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:43.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 shutdown_connections 2024-08-20T06:35:43.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:43.801+0000 7f416fbc2640 1 -- 172.21.15.107:0/1061424140 wait complete. 2024-08-20T06:35:44.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 -- 172.21.15.107:0/3466039798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401039d0 msgr2=0x7f8840103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 --2- 172.21.15.107:0/3466039798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401039d0 0x7f8840103db0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8834009900 tx=0x7f883402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 -- 172.21.15.107:0/3466039798 shutdown_connections 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 --2- 172.21.15.107:0/3466039798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884010ce30 0x7f884010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 --2- 172.21.15.107:0/3466039798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88401042f0 0x7f884010c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 --2- 172.21.15.107:0/3466039798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401039d0 0x7f8840103db0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 -- 172.21.15.107:0/3466039798 >> 172.21.15.107:0/3466039798 conn(0x7f88400fd410 msgr2=0x7f88400ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 -- 172.21.15.107:0/3466039798 shutdown_connections 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.317+0000 7f8847170640 1 -- 172.21.15.107:0/3466039798 wait complete. 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 Processor -- start 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 -- start start 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88401039d0 0x7f8840199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 0x7f88401997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884010ce30 0x7f884019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88401129d0 con 0x7f88401042f0 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8840112850 con 0x7f88401039d0 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.318+0000 7f8847170640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8840112b50 con 0x7f884010ce30 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 0x7f88401997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 0x7f88401997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40746/0 (socket says 172.21.15.107:40746) 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 -- 172.21.15.107:0/2237209640 learned_addr learned my addr 172.21.15.107:0/2237209640 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f8844f0c640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88401039d0 0x7f8840199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f884570d640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884010ce30 0x7f884019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 -- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884010ce30 msgr2=0x7f884019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f884010ce30 0x7f884019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 -- 172.21.15.107:0/2237209640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88401039d0 msgr2=0x7f8840199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88401039d0 0x7f8840199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884019f330 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883ffff640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 0x7f88401997c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b3c0 tx=0x7f882c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c011830 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884019f620 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f882c016070 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f882c010d20 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.319+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88401a5eb0 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.320+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f884006b8b0 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.322+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f882c01b460 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.322+0000 7f883dffb640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88300b3c60 0x7f88300b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f8844f0c640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88300b3c60 0x7f88300b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f882c0e3a50 con 0x7f88401042f0 2024-08-20T06:35:44.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f883dffb640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] conn(0x7f88300bd4b0 0x7f88300bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f882c022e50 con 0x7f88300bd4b0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f882c0e3f30 con 0x7f88401042f0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f8844f0c640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88300b3c60 0x7f88300b6120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88340057c0 tx=0x7f88340058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.323+0000 7f884570d640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] conn(0x7f88300bd4b0 0x7f88300bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.324+0000 7f884570d640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] conn(0x7f88300bd4b0 0x7f88300bf8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.324+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== osd.8 v2:172.21.15.193:6816/2908108197 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f882c022e50 con 0x7f88300bd4b0 2024-08-20T06:35:44.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.338+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 --> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8840063510 con 0x7f88300bd4b0 2024-08-20T06:35:44.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f883dffb640 1 -- 172.21.15.107:0/2237209640 <== osd.8 v2:172.21.15.193:6816/2908108197 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8840063510 con 0x7f88300bd4b0 2024-08-20T06:35:44.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] conn(0x7f88300bd4b0 msgr2=0x7f88300bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] conn(0x7f88300bd4b0 0x7f88300bf8d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88300b3c60 msgr2=0x7f88300b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88300b3c60 0x7f88300b6120 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f88340057c0 tx=0x7f88340058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 msgr2=0x7f88401997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 --2- 172.21.15.107:0/2237209640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f88401042f0 0x7f88401997c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f882c00b3c0 tx=0x7f882c00b880 comp rx=0 tx=0).stop 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f884570d640 1 -- 172.21.15.107:0/2237209640 reap_dead start 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 shutdown_connections 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 >> 172.21.15.107:0/2237209640 conn(0x7f88400fd410 msgr2=0x7f88400fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 shutdown_connections 2024-08-20T06:35:44.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.339+0000 7f8847170640 1 -- 172.21.15.107:0/2237209640 wait complete. 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 -- 172.21.15.107:0/3719045048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e64101520 msgr2=0x7f5e64101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/3719045048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e64101520 0x7f5e64101900 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5e58009900 tx=0x7f5e5802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 -- 172.21.15.107:0/3719045048 shutdown_connections 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/3719045048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e641074d0 0x7f5e6410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/3719045048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e64106b50 0x7f5e64106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/3719045048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e64101520 0x7f5e64101900 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 -- 172.21.15.107:0/3719045048 >> 172.21.15.107:0/3719045048 conn(0x7f5e64076c80 msgr2=0x7f5e64077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 -- 172.21.15.107:0/3719045048 shutdown_connections 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.535+0000 7f5e6afa1640 1 -- 172.21.15.107:0/3719045048 wait complete. 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 Processor -- start 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 -- start start 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e64101520 0x7f5e64199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e64106b50 0x7f5e6419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 0x7f5e6419a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64110fb0 con 0x7f5e641074d0 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64110e30 con 0x7f5e64106b50 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6afa1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5e64111130 con 0x7f5e64101520 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6953e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 0x7f5e6419a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6953e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 0x7f5e6419a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40752/0 (socket says 172.21.15.107:40752) 2024-08-20T06:35:44.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e6953e640 1 -- 172.21.15.107:0/2519875144 learned_addr learned my addr 172.21.15.107:0/2519875144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e63fff640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e64106b50 0x7f5e6419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.536+0000 7f5e68d3d640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e64101520 0x7f5e64199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 -- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e64101520 msgr2=0x7f5e64199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e64101520 0x7f5e64199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 -- 172.21.15.107:0/2519875144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e64106b50 msgr2=0x7f5e6419a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5e64106b50 0x7f5e6419a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641a5070 con 0x7f5e641074d0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6953e640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 0x7f5e6419a730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00cad0 tx=0x7f5e5c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00ad60 con 0x7f5e641074d0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5e5c005020 con 0x7f5e641074d0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c00f6b0 con 0x7f5e641074d0 2024-08-20T06:35:44.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641a5360 con 0x7f5e641074d0 2024-08-20T06:35:44.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.537+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e641a58f0 con 0x7f5e641074d0 2024-08-20T06:35:44.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.538+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e64105350 con 0x7f5e641074d0 2024-08-20T06:35:44.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.539+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f5e5c0051d0 con 0x7f5e641074d0 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e61ffb640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e540b3ce0 0x7f5e540b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e68d3d640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e540b3ce0 0x7f5e540b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f5e5c0e3f50 con 0x7f5e641074d0 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e61ffb640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] conn(0x7f5e540bd530 0x7f5e540bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e5c016ec0 con 0x7f5e540bd530 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.540+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f5e5c0e4430 con 0x7f5e641074d0 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.541+0000 7f5e68d3d640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e540b3ce0 0x7f5e540b61a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000c00 tx=0x7f5e58005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.541+0000 7f5e63fff640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] conn(0x7f5e540bd530 0x7f5e540bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.541+0000 7f5e63fff640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] conn(0x7f5e540bd530 0x7f5e540bf950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:44.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.542+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== osd.2 v2:172.21.15.193:6800/4087892550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e5c016ec0 con 0x7f5e540bd530 2024-08-20T06:35:44.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.556+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 --> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5e641024e0 con 0x7f5e540bd530 2024-08-20T06:35:44.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e61ffb640 1 -- 172.21.15.107:0/2519875144 <== osd.2 v2:172.21.15.193:6800/4087892550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5e641024e0 con 0x7f5e540bd530 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] conn(0x7f5e540bd530 msgr2=0x7f5e540bf950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] conn(0x7f5e540bd530 0x7f5e540bf950 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e540b3ce0 msgr2=0x7f5e540b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5e540b3ce0 0x7f5e540b61a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000c00 tx=0x7f5e58005a20 comp rx=0 tx=0).stop 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 msgr2=0x7f5e6419a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 --2- 172.21.15.107:0/2519875144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5e641074d0 0x7f5e6419a730 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c00cad0 tx=0x7f5e5c00ced0 comp rx=0 tx=0).stop 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6953e640 1 -- 172.21.15.107:0/2519875144 reap_dead start 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 shutdown_connections 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.557+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 >> 172.21.15.107:0/2519875144 conn(0x7f5e64076c80 msgr2=0x7f5e641048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.558+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 shutdown_connections 2024-08-20T06:35:44.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:44.558+0000 7f5e6afa1640 1 -- 172.21.15.107:0/2519875144 wait complete. 2024-08-20T06:35:45.013 INFO:teuthology.orchestra.run.smithi107.stdout:146028888114 2024-08-20T06:35:45.013 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-20T06:35:45.356 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:45 smithi193 ceph-mon[21944]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:35:45.358 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:45 smithi107 ceph-mon[18495]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:35:45.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- 172.21.15.107:0/3369971909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f658810cc60 msgr2=0x7f658810f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 --2- 172.21.15.107:0/3369971909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f658810cc60 0x7f658810f0c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6580009ca0 tx=0x7f658002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- 172.21.15.107:0/3369971909 shutdown_connections 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 --2- 172.21.15.107:0/3369971909 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f658810cc60 0x7f658810f0c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 --2- 172.21.15.107:0/3369971909 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 0x7f658810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 --2- 172.21.15.107:0/3369971909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6588103800 0x7f6588103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- 172.21.15.107:0/3369971909 >> 172.21.15.107:0/3369971909 conn(0x7f65880fd2a0 msgr2=0x7f65880ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- 172.21.15.107:0/3369971909 shutdown_connections 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- 172.21.15.107:0/3369971909 wait complete. 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 Processor -- start 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.011+0000 7f658d7a1640 1 -- start start 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 0x7f6588198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 0x7f6588199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658810cc60 0x7f658819e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6588112810 con 0x7f6588103800 2024-08-20T06:35:45.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6588112690 con 0x7f6588104120 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658d7a1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6588112990 con 0x7f658810cc60 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 0x7f6588198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 0x7f6588198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40700/0 (socket says 172.21.15.107:40700) 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 -- 172.21.15.107:0/4291683892 learned_addr learned my addr 172.21.15.107:0/4291683892 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f65877fe640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 0x7f6588199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f658cfa0640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658810cc60 0x7f658819e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 -- 172.21.15.107:0/4291683892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658810cc60 msgr2=0x7f658819e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f658810cc60 0x7f658819e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 -- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 msgr2=0x7f6588199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 0x7f6588199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f6587fff640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658819ecd0 con 0x7f6588103800 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.012+0000 7f65877fe640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6588104120 0x7f6588199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f6587fff640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 0x7f6588198c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f657800b4a0 tx=0x7f657800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:45.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578014070 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658819efc0 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6578010aa0 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578017b70 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65881a5680 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.013+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65880ff200 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.015+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f6578017070 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.015+0000 7f65857fa640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f65700b3d10 0x7f65700b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65877fe640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f65700b3d10 0x7f65700b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f65780e3360 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65857fa640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] conn(0x7f65700bd560 0x7f65700bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65700c0030 con 0x7f65700bd560 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f65780e3840 con 0x7f6588103800 2024-08-20T06:35:45.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f65877fe640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f65700b3d10 0x7f65700b61d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f658810e290 tx=0x7f657c0092a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.016+0000 7f658cfa0640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] conn(0x7f65700bd560 0x7f65700bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.017+0000 7f658cfa0640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] conn(0x7f65700bd560 0x7f65700bf980 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.017+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== osd.7 v2:172.21.15.183:6816/3897620215 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65700c0030 con 0x7f65700bd560 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.031+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 --> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f658806b6d0 con 0x7f65700bd560 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.031+0000 7f65857fa640 1 -- 172.21.15.107:0/4291683892 <== osd.7 v2:172.21.15.183:6816/3897620215 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f658806b6d0 con 0x7f65700bd560 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] conn(0x7f65700bd560 msgr2=0x7f65700bf980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] conn(0x7f65700bd560 0x7f65700bf980 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f65700b3d10 msgr2=0x7f65700b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f65700b3d10 0x7f65700b61d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f658810e290 tx=0x7f657c0092a0 comp rx=0 tx=0).stop 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 msgr2=0x7f6588198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 --2- 172.21.15.107:0/4291683892 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6588103800 0x7f6588198c20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f657800b4a0 tx=0x7f657800b960 comp rx=0 tx=0).stop 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658cfa0640 1 -- 172.21.15.107:0/4291683892 reap_dead start 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 shutdown_connections 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 >> 172.21.15.107:0/4291683892 conn(0x7f65880fd2a0 msgr2=0x7f658810eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 shutdown_connections 2024-08-20T06:35:45.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:42.032+0000 7f658d7a1640 1 -- 172.21.15.107:0/4291683892 wait complete. 2024-08-20T06:35:45.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:45 smithi183 ceph-mon[22204]: pgmap v292: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:35:46.164 INFO:teuthology.orchestra.run.smithi107.stdout:193273528360 2024-08-20T06:35:46.164 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-20T06:35:46.950 INFO:teuthology.orchestra.run.smithi107.stdout:219043332133 2024-08-20T06:35:46.950 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-20T06:35:47.323 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:47 smithi107 ceph-mon[18495]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:47.351 INFO:teuthology.orchestra.run.smithi107.stdout:279172874261 2024-08-20T06:35:47.351 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-20T06:35:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:47 smithi183 ceph-mon[22204]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:47 smithi193 ceph-mon[21944]: pgmap v293: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 -- 172.21.15.107:0/2322970716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108500 msgr2=0x7ffab8108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 --2- 172.21.15.107:0/2322970716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108500 0x7ffab8108940 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009900 tx=0x7ffaac02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 -- 172.21.15.107:0/2322970716 shutdown_connections 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 --2- 172.21.15.107:0/2322970716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffab8108e80 0x7ffab810f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 --2- 172.21.15.107:0/2322970716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108500 0x7ffab8108940 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 --2- 172.21.15.107:0/2322970716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab8102af0 0x7ffab8102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 -- 172.21.15.107:0/2322970716 >> 172.21.15.107:0/2322970716 conn(0x7ffab80fe540 msgr2=0x7ffab8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 -- 172.21.15.107:0/2322970716 shutdown_connections 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.106+0000 7ffabfb21640 1 -- 172.21.15.107:0/2322970716 wait complete. 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 Processor -- start 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 -- start start 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab8102af0 0x7ffab8198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffab8108500 0x7ffab8199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 0x7ffab819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8112980 con 0x7ffab8108e80 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8112800 con 0x7ffab8108500 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabfb21640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffab8112b00 con 0x7ffab8102af0 2024-08-20T06:35:48.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabe0be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 0x7ffab819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:48.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabe0be640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 0x7ffab819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53022/0 (socket says 172.21.15.107:53022) 2024-08-20T06:35:48.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabe0be640 1 -- 172.21.15.107:0/1576967073 learned_addr learned my addr 172.21.15.107:0/1576967073 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:48.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabd8bd640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab8102af0 0x7ffab8198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:48.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.107+0000 7ffabd0bc640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffab8108500 0x7ffab8199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:48.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 -- 172.21.15.107:0/1576967073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab8102af0 msgr2=0x7ffab8198e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab8102af0 0x7ffab8198e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 -- 172.21.15.107:0/1576967073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffab8108500 msgr2=0x7ffab8199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffab8108500 0x7ffab8199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab819eec0 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabe0be640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 0x7ffab819e6f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e970 tx=0x7ffab000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab000ccc0 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffab000ce60 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab819f1b0 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab0010430 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.108+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab81a5870 con 0x7ffab8108e80 2024-08-20T06:35:48.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.109+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffab8063610 con 0x7ffab8108e80 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.110+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7ffab0004ef0 con 0x7ffab8108e80 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffab6ffd640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffaa00b3ce0 0x7ffaa00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffabd8bd640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffaa00b3ce0 0x7ffaa00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7ffab00e4390 con 0x7ffab8108e80 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffab6ffd640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] conn(0x7ffaa00bd530 0x7ffaa00bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffab0016ec0 con 0x7ffaa00bd530 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.111+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7ffab00e4870 con 0x7ffab8108e80 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.112+0000 7ffabd0bc640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] conn(0x7ffaa00bd530 0x7ffaa00bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.112+0000 7ffabd8bd640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffaa00b3ce0 0x7ffaa00b61a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007ec0 tx=0x7ffaa80075b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:48.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.112+0000 7ffabd0bc640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] conn(0x7ffaa00bd530 0x7ffaa00bf950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.112+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== osd.9 v2:172.21.15.107:6826/1423000516 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffab0016ec0 con 0x7ffaa00bd530 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.126+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 --> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffab8063a90 con 0x7ffaa00bd530 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.126+0000 7ffab6ffd640 1 -- 172.21.15.107:0/1576967073 <== osd.9 v2:172.21.15.107:6826/1423000516 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffab8063a90 con 0x7ffaa00bd530 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] conn(0x7ffaa00bd530 msgr2=0x7ffaa00bf950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] conn(0x7ffaa00bd530 0x7ffaa00bf950 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffaa00b3ce0 msgr2=0x7ffaa00b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffaa00b3ce0 0x7ffaa00b61a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8007ec0 tx=0x7ffaa80075b0 comp rx=0 tx=0).stop 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 msgr2=0x7ffab819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 --2- 172.21.15.107:0/1576967073 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffab8108e80 0x7ffab819e6f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ffab000e970 tx=0x7ffab000ed70 comp rx=0 tx=0).stop 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabe0be640 1 -- 172.21.15.107:0/1576967073 reap_dead start 2024-08-20T06:35:48.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 shutdown_connections 2024-08-20T06:35:48.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 >> 172.21.15.107:0/1576967073 conn(0x7ffab80fe540 msgr2=0x7ffab8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:48.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 shutdown_connections 2024-08-20T06:35:48.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:48.127+0000 7ffabfb21640 1 -- 172.21.15.107:0/1576967073 wait complete. 2024-08-20T06:35:48.889 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:48.914 INFO:teuthology.orchestra.run.smithi107.stdout:73014444097 2024-08-20T06:35:48.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-20T06:35:49.147 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:49 smithi107 ceph-mon[18495]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:49.413 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.432 INFO:teuthology.orchestra.run.smithi107.stdout:240518168608 2024-08-20T06:35:49.432 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-20T06:35:49.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:49 smithi183 ceph-mon[22204]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:49 smithi193 ceph-mon[21944]: pgmap v294: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.679+0000 7f9407276640 1 -- 172.21.15.107:0/2888852481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 msgr2=0x7f9400101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.679+0000 7f9407276640 1 --2- 172.21.15.107:0/2888852481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400101d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009900 tx=0x7f93f002a410 comp rx=0 tx=0).stop 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 -- 172.21.15.107:0/2888852481 shutdown_connections 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 --2- 172.21.15.107:0/2888852481 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 --2- 172.21.15.107:0/2888852481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 0x7f94001077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 --2- 172.21.15.107:0/2888852481 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400101d70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 -- 172.21.15.107:0/2888852481 >> 172.21.15.107:0/2888852481 conn(0x7f94000fd400 msgr2=0x7f94000ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:49.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 -- 172.21.15.107:0/2888852481 shutdown_connections 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 -- 172.21.15.107:0/2888852481 wait complete. 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 Processor -- start 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.680+0000 7f9407276640 1 -- start start 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 0x7f94001994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9400111890 con 0x7f9400101990 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9400111710 con 0x7f9400107d20 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9407276640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9400111a10 con 0x7f94001073a0 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:49.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53038/0 (socket says 172.21.15.107:53038) 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 -- 172.21.15.107:0/2577955816 learned_addr learned my addr 172.21.15.107:0/2577955816 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405813640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 -- 172.21.15.107:0/2577955816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 msgr2=0x7f94001994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9404811640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 0x7f94001994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 0x7f94001994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 -- 172.21.15.107:0/2577955816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 msgr2=0x7f940019e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940019e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 -- 172.21.15.107:0/2577955816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f940019f020 con 0x7f9400101990 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405813640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f9405012640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009a20 tx=0x7f93f0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f003f030 con 0x7f9400101990 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93f00035f0 con 0x7f9400101990 2024-08-20T06:35:49.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.681+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0035070 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.682+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f940019f2b0 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.682+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94001a59d0 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.682+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9400101d70 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.685+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f93f0003aa0 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.686+0000 7f93f67fc640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 0x7f93ec0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.686+0000 7f9404811640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 0x7f93ec0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.686+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f93f0104b80 con 0x7f9400101990 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.686+0000 7f9404811640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 0x7f93ec0b61a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9400106310 tx=0x7f93fc005c60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:49.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.689+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f93f00cc220 con 0x7f9400101990 2024-08-20T06:35:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.775 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.786 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.795 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.805 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:49.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.910+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f940010a580 con 0x7f9400101990 2024-08-20T06:35:49.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.911+0000 7f93f67fc640 1 -- 172.21.15.107:0/2577955816 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f93f00cbb20 con 0x7f9400101990 2024-08-20T06:35:49.912 INFO:teuthology.orchestra.run.smithi107.stdout:98784247867 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 msgr2=0x7f93ec0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 0x7f93ec0b61a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9400106310 tx=0x7f93fc005c60 comp rx=0 tx=0).stop 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 msgr2=0x7f9400198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f93f0009a20 tx=0x7f93f0004270 comp rx=0 tx=0).stop 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 shutdown_connections 2024-08-20T06:35:49.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9400107d20 0x7f940019e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93ec0b3ce0 0x7f93ec0b61a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94001073a0 0x7f94001994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 --2- 172.21.15.107:0/2577955816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9400101990 0x7f9400198f70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 >> 172.21.15.107:0/2577955816 conn(0x7f94000fd400 msgr2=0x7f940006ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 shutdown_connections 2024-08-20T06:35:49.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:49.913+0000 7f9407276640 1 -- 172.21.15.107:0/2577955816 wait complete. 2024-08-20T06:35:50.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:50 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2577955816' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T06:35:50.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:50 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2577955816' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T06:35:50.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:50 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2577955816' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T06:35:51.139 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:51 smithi107 ceph-mon[18495]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-20T06:35:51.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 -- 172.21.15.107:0/2126905648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe58107d60 msgr2=0x7fbe5810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/2126905648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe58107d60 0x7fbe5810e560 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbe50009ca0 tx=0x7fbe5002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 -- 172.21.15.107:0/2126905648 shutdown_connections 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/2126905648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe58107d60 0x7fbe5810e560 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/2126905648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe581073e0 0x7fbe58107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/2126905648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe581019d0 0x7fbe58101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 -- 172.21.15.107:0/2126905648 >> 172.21.15.107:0/2126905648 conn(0x7fbe580fd440 msgr2=0x7fbe580ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 -- 172.21.15.107:0/2126905648 shutdown_connections 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.281+0000 7fbe5ecae640 1 -- 172.21.15.107:0/2126905648 wait complete. 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 Processor -- start 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 -- start start 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe581019d0 0x7fbe5819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 0x7fbe5819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:51.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe58107d60 0x7fbe581a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58112a20 con 0x7fbe581073e0 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe581128a0 con 0x7fbe58107d60 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ecae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe58112ba0 con 0x7fbe581019d0 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 0x7fbe5819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 0x7fbe5819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53066/0 (socket says 172.21.15.107:53066) 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 -- 172.21.15.107:0/1984844585 learned_addr learned my addr 172.21.15.107:0/1984844585 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5ca4a640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe581019d0 0x7fbe5819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe5d24b640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe58107d60 0x7fbe581a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:51.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 -- 172.21.15.107:0/1984844585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe581019d0 msgr2=0x7fbe5819b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe581019d0 0x7fbe5819b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 -- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe58107d60 msgr2=0x7fbe581a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe58107d60 0x7fbe581a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.282+0000 7fbe57fff640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe581a1160 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe57fff640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 0x7fbe5819b5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d950 tx=0x7fbe4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44017070 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe44009c30 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe581a1450 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400f620 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.283+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe581a7b10 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.284+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe5810fa60 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.285+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fbe44010460 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe55ffb640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe480b3c60 0x7fbe480b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe5ca4a640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe480b3c60 0x7fbe480b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fbe440e3ba0 con 0x7fbe581073e0 2024-08-20T06:35:51.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe55ffb640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] conn(0x7fbe480bd4b0 0x7fbe480bf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe44016ec0 con 0x7fbe480bd4b0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7fbe440e4080 con 0x7fbe581073e0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.286+0000 7fbe5ca4a640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe480b3c60 0x7fbe480b6120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005fd0 tx=0x7fbe4c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.287+0000 7fbe5d24b640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] conn(0x7fbe480bd4b0 0x7fbe480bf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.287+0000 7fbe5d24b640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] conn(0x7fbe480bd4b0 0x7fbe480bf8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.288+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== osd.4 v2:172.21.15.183:6808/3081917594 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe44016ec0 con 0x7fbe480bd4b0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.301+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 --> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe58102c80 con 0x7fbe480bd4b0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe55ffb640 1 -- 172.21.15.107:0/1984844585 <== osd.4 v2:172.21.15.183:6808/3081917594 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe58102c80 con 0x7fbe480bd4b0 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] conn(0x7fbe480bd4b0 msgr2=0x7fbe480bf8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] conn(0x7fbe480bd4b0 0x7fbe480bf8d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe480b3c60 msgr2=0x7fbe480b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe480b3c60 0x7fbe480b6120 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c005fd0 tx=0x7fbe4c00b390 comp rx=0 tx=0).stop 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 msgr2=0x7fbe5819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:51.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 --2- 172.21.15.107:0/1984844585 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe581073e0 0x7fbe5819b5f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400d950 tx=0x7fbe4400de10 comp rx=0 tx=0).stop 2024-08-20T06:35:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5d24b640 1 -- 172.21.15.107:0/1984844585 reap_dead start 2024-08-20T06:35:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 shutdown_connections 2024-08-20T06:35:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 >> 172.21.15.107:0/1984844585 conn(0x7fbe580fd440 msgr2=0x7fbe5810f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 shutdown_connections 2024-08-20T06:35:51.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:51.302+0000 7fbe5ecae640 1 -- 172.21.15.107:0/1984844585 wait complete. 2024-08-20T06:35:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:51 smithi183 ceph-mon[22204]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-20T06:35:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:51 smithi193 ceph-mon[21944]: pgmap v295: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-08-20T06:35:52.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 -- 172.21.15.107:0/1935615502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c107d20 msgr2=0x7f7e3c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 --2- 172.21.15.107:0/1935615502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c107d20 0x7f7e3c10e520 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009ca0 tx=0x7f7e3402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 -- 172.21.15.107:0/1935615502 shutdown_connections 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 --2- 172.21.15.107:0/1935615502 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c107d20 0x7f7e3c10e520 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 --2- 172.21.15.107:0/1935615502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 --2- 172.21.15.107:0/1935615502 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 -- 172.21.15.107:0/1935615502 >> 172.21.15.107:0/1935615502 conn(0x7f7e3c0fd400 msgr2=0x7f7e3c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 -- 172.21.15.107:0/1935615502 shutdown_connections 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.147+0000 7f7e40827640 1 -- 172.21.15.107:0/1935615502 wait complete. 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 Processor -- start 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 -- start start 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 0x7f7e3c19e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c1118a0 con 0x7f7e3c1073a0 2024-08-20T06:35:52.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111720 con 0x7f7e3c101990 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e40827640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7e3c111a20 con 0x7f7e3c107d20 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53104/0 (socket says 172.21.15.107:53104) 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 -- 172.21.15.107:0/1657374798 learned_addr learned my addr 172.21.15.107:0/1657374798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e3a59c640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e3ad9d640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 0x7f7e3c19e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 -- 172.21.15.107:0/1657374798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 msgr2=0x7f7e3c19e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 0x7f7e3c19e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 -- 172.21.15.107:0/1657374798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 msgr2=0x7f7e3c198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 -- 172.21.15.107:0/1657374798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e3c19ee90 con 0x7f7e3c1073a0 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e3a59c640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e3ad9d640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 0x7f7e3c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:52.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.148+0000 7f7e39d9b640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d950 tx=0x7f7e3000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.149+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e30017070 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.149+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e3c19f180 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.149+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7e30009c30 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.149+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000f620 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.149+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e3c1a5a10 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.150+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e3c102c40 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.151+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f7e30010460 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.152+0000 7f7e2b7fe640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 0x7f7e240b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.152+0000 7f7e3a59c640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 0x7f7e240b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.152+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f7e300e3900 con 0x7f7e3c1073a0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.152+0000 7f7e3a59c640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 0x7f7e240b6120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00abe0 tx=0x7f7e2c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:52.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.155+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7e300aafa0 con 0x7f7e3c1073a0 2024-08-20T06:35:52.320 INFO:teuthology.orchestra.run.smithi107.stdout:124554051641 2024-08-20T06:35:52.321 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-20T06:35:52.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.344+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7e3c1a5be0 con 0x7f7e3c1073a0 2024-08-20T06:35:52.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.345+0000 7f7e2b7fe640 1 -- 172.21.15.107:0/1657374798 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7e30009de0 con 0x7f7e3c1073a0 2024-08-20T06:35:52.345 INFO:teuthology.orchestra.run.smithi107.stdout:257698037787 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 msgr2=0x7f7e240b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 0x7f7e240b6120 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7e2c00abe0 tx=0x7f7e2c00a5c0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 msgr2=0x7f7e3c199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000d950 tx=0x7f7e3000de10 comp rx=0 tx=0).stop 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 shutdown_connections 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7e240b3c60 0x7f7e240b6120 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e3c107d20 0x7f7e3c19e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7e3c1073a0 0x7f7e3c199320 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 --2- 172.21.15.107:0/1657374798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7e3c101990 0x7f7e3c198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 >> 172.21.15.107:0/1657374798 conn(0x7f7e3c0fd400 msgr2=0x7f7e3c105720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:52.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 shutdown_connections 2024-08-20T06:35:52.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.347+0000 7f7e40827640 1 -- 172.21.15.107:0/1657374798 wait complete. 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- 172.21.15.107:0/2302719851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330104270 msgr2=0x7f033010c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 --2- 172.21.15.107:0/2302719851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330104270 0x7f033010c850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0318009900 tx=0x7f031802a410 comp rx=0 tx=0).stop 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- 172.21.15.107:0/2302719851 shutdown_connections 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 --2- 172.21.15.107:0/2302719851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f033010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 --2- 172.21.15.107:0/2302719851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330104270 0x7f033010c850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 --2- 172.21.15.107:0/2302719851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330103870 0x7f0330103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- 172.21.15.107:0/2302719851 >> 172.21.15.107:0/2302719851 conn(0x7f03300fd440 msgr2=0x7f03300ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- 172.21.15.107:0/2302719851 shutdown_connections 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- 172.21.15.107:0/2302719851 wait complete. 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 Processor -- start 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.049+0000 7f0335469640 1 -- start start 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330104270 0x7f0330101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f0330102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112950 con 0x7f0330103870 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f03301127d0 con 0x7f0330104270 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0330112ad0 con 0x7f033010ce20 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53078/0 (socket says 172.21.15.107:53078) 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 -- 172.21.15.107:0/1396973876 learned_addr learned my addr 172.21.15.107:0/1396973876 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032f7fe640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f0330102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032e7fc640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330104270 0x7f0330101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 -- 172.21.15.107:0/1396973876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 msgr2=0x7f0330102170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f0330102170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 -- 172.21.15.107:0/1396973876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330104270 msgr2=0x7f0330101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330104270 0x7f0330101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 -- 172.21.15.107:0/1396973876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03301a72a0 con 0x7f0330103870 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stdout:146028888116 2024-08-20T06:35:52.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032f7fe640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f0330102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f032effd640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f032000d950 tx=0x7f032000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0320017030 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301a7590 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0320004020 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03200055d0 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.050+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301a7b20 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.051+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330103c50 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.053+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f0320009a30 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.053+0000 7f0327fff640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 0x7f031c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.054+0000 7f032e7fc640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 0x7f031c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.054+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f03200e3db0 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.054+0000 7f032e7fc640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 0x7f031c0b6120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0318006010 tx=0x7f0318000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.056+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f03200ab450 con 0x7f0330103870 2024-08-20T06:35:52.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.245+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f03301029f0 con 0x7f0330103870 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.246+0000 7f0327fff640 1 -- 172.21.15.107:0/1396973876 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f03200aad50 con 0x7f0330103870 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 msgr2=0x7f031c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 0x7f031c0b6120 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0318006010 tx=0x7f0318000bc0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 msgr2=0x7f03301016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f032000d950 tx=0x7f032000de10 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 shutdown_connections 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f031c0b3c60 0x7f031c0b6120 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f033010ce20 0x7f0330102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0330104270 0x7f0330101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 --2- 172.21.15.107:0/1396973876 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0330103870 0x7f03301016f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 >> 172.21.15.107:0/1396973876 conn(0x7f03300fd440 msgr2=0x7f03300fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:52.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 shutdown_connections 2024-08-20T06:35:52.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:52.248+0000 7f0335469640 1 -- 172.21.15.107:0/1396973876 wait complete. 2024-08-20T06:35:53.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:53 smithi107 ceph-mon[18495]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-20T06:35:53.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:53 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1396973876' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T06:35:53.257 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:53 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1657374798' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T06:35:53.284 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037785 got 257698037787 for osd.10 2024-08-20T06:35:53.284 DEBUG:teuthology.parallel:result is None 2024-08-20T06:35:53.404 INFO:tasks.cephadm.ceph_manager.ceph:need seq 98784247866 got 98784247867 for osd.3 2024-08-20T06:35:53.404 DEBUG:teuthology.parallel:result is None 2024-08-20T06:35:53.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:53 smithi183 ceph-mon[22204]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-20T06:35:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:53 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1396973876' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T06:35:53.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:53 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1657374798' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T06:35:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:53 smithi193 ceph-mon[21944]: pgmap v296: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s 2024-08-20T06:35:53.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:53 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1396973876' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T06:35:53.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:53 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1657374798' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T06:35:53.831 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888114 got 146028888116 for osd.5 2024-08-20T06:35:53.831 DEBUG:teuthology.parallel:result is None 2024-08-20T06:35:54.045 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:54.046 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:54.049 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:54.049 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:54.052 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.507+0000 7fae151c3640 1 -- 172.21.15.107:0/3465453694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 msgr2=0x7fae10103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.507+0000 7fae151c3640 1 --2- 172.21.15.107:0/3465453694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10103c30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fae00009ca0 tx=0x7fae0002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 -- 172.21.15.107:0/3465453694 shutdown_connections 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 --2- 172.21.15.107:0/3465453694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae1010ce00 0x7fae1010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 --2- 172.21.15.107:0/3465453694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae10104250 0x7fae1010c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 --2- 172.21.15.107:0/3465453694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10103c30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 -- 172.21.15.107:0/3465453694 >> 172.21.15.107:0/3465453694 conn(0x7fae100fd400 msgr2=0x7fae100ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 -- 172.21.15.107:0/3465453694 shutdown_connections 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 -- 172.21.15.107:0/3465453694 wait complete. 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.508+0000 7fae151c3640 1 Processor -- start 2024-08-20T06:35:54.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 -- start start 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 0x7fae10199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae1010ce00 0x7fae1019e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae10112a20 con 0x7fae10103850 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae101128a0 con 0x7fae10104250 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae151c3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae10112ba0 con 0x7fae1010ce00 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53116/0 (socket says 172.21.15.107:53116) 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 -- 172.21.15.107:0/919903778 learned_addr learned my addr 172.21.15.107:0/919903778 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0f59e640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae1010ce00 0x7fae1019e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0e59c640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 0x7fae10199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 -- 172.21.15.107:0/919903778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae1010ce00 msgr2=0x7fae1019e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae1010ce00 0x7fae1019e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 -- 172.21.15.107:0/919903778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 msgr2=0x7fae10199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:54.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 0x7fae10199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0ed9d640 1 -- 172.21.15.107:0/919903778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae1019f0c0 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.509+0000 7fae0e59c640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 0x7fae10199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae0ed9d640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae00006010 tx=0x7fae00032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00004040 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae1019f350 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae00004670 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00030da0 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.510+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae101a5a70 con 0x7fae10103850 2024-08-20T06:35:54.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.511+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae10103c30 con 0x7fae10103850 2024-08-20T06:35:54.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.513+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fae000305c0 con 0x7fae10103850 2024-08-20T06:35:54.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.514+0000 7fae07fff640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 0x7fadfc0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:54.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.514+0000 7fae0e59c640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 0x7fadfc0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:54.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.514+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fae001053f0 con 0x7fae10103850 2024-08-20T06:35:54.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.514+0000 7fae0e59c640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 0x7fadfc0b6120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae1010f3f0 tx=0x7fadf8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:54.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.517+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fae00004820 con 0x7fae10103850 2024-08-20T06:35:54.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.698+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fae101078e0 con 0x7fae10103850 2024-08-20T06:35:54.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.698+0000 7fae07fff640 1 -- 172.21.15.107:0/919903778 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fae000d3e20 con 0x7fae10103850 2024-08-20T06:35:54.699 INFO:teuthology.orchestra.run.smithi107.stdout:193273528363 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 msgr2=0x7fadfc0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 0x7fadfc0b6120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fae1010f3f0 tx=0x7fadf8008040 comp rx=0 tx=0).stop 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 msgr2=0x7fae10199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fae00006010 tx=0x7fae00032bc0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 shutdown_connections 2024-08-20T06:35:54.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fadfc0b3c60 0x7fadfc0b6120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae1010ce00 0x7fae1019e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae10104250 0x7fae10199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 --2- 172.21.15.107:0/919903778 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae10103850 0x7fae10199010 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.700+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 >> 172.21.15.107:0/919903778 conn(0x7fae100fd400 msgr2=0x7fae100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.701+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 shutdown_connections 2024-08-20T06:35:54.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:54.701+0000 7fae151c3640 1 -- 172.21.15.107:0/919903778 wait complete. 2024-08-20T06:35:55.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:55 smithi107 ceph-mon[18495]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:35:55.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:55 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/919903778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T06:35:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:55 smithi183 ceph-mon[22204]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:35:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:55 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/919903778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T06:35:55.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 193273528360 got 193273528363 for osd.7 2024-08-20T06:35:55.525 DEBUG:teuthology.parallel:result is None 2024-08-20T06:35:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:55 smithi193 ceph-mon[21944]: pgmap v297: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:35:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:55 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/919903778' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/32724267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8108e60 msgr2=0x7f2dc810f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/32724267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8108e60 0x7f2dc810f660 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009ca0 tx=0x7f2dc002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/32724267 shutdown_connections 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/32724267 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8108e60 0x7f2dc810f660 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/32724267 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 0x7f2dc8108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/32724267 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/32724267 >> 172.21.15.107:0/32724267 conn(0x7f2dc80fe540 msgr2=0x7f2dc8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/32724267 shutdown_connections 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/32724267 wait complete. 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 Processor -- start 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- start start 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 0x7f2dc81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8108e60 0x7f2dc819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc81129e0 con 0x7f2dc8102ad0 2024-08-20T06:35:55.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8112860 con 0x7f2dc8108e60 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.953+0000 7f2dcdb6f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc8112b60 con 0x7f2dc81084e0 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60676/0 (socket says 172.21.15.107:60676) 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 -- 172.21.15.107:0/3338319114 learned_addr learned my addr 172.21.15.107:0/3338319114 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dbffff640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 0x7f2dc81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc7fff640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8108e60 0x7f2dc819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 -- 172.21.15.107:0/3338319114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 msgr2=0x7f2dc81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 0x7f2dc81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 -- 172.21.15.107:0/3338319114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8108e60 msgr2=0x7f2dc819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:55.965 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8108e60 0x7f2dc819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 -- 172.21.15.107:0/3338319114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc819ef40 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc77fe640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2db800d950 tx=0x7f2db800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8017030 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2db8004020 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc819f230 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db80055d0 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.954+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc81a58f0 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.955+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc8102eb0 con 0x7f2dc8102ad0 2024-08-20T06:35:55.966 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.957+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2db8009a30 con 0x7f2dc8102ad0 2024-08-20T06:35:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.957+0000 7f2dc57fa640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 0x7f2db40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.957+0000 7f2dbffff640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 0x7f2db40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.957+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f2db80e3c00 con 0x7f2dc8102ad0 2024-08-20T06:35:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.958+0000 7f2dbffff640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 0x7f2db40b6120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8104eb0 tx=0x7f2db000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:55.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:55.960+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2db80ab2a0 con 0x7f2dc8102ad0 2024-08-20T06:35:56.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.143+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f2dc802e340 con 0x7f2dc8102ad0 2024-08-20T06:35:56.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.144+0000 7f2dc57fa640 1 -- 172.21.15.107:0/3338319114 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2db80aaba0 con 0x7f2dc8102ad0 2024-08-20T06:35:56.145 INFO:teuthology.orchestra.run.smithi107.stdout:240518168609 2024-08-20T06:35:56.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 msgr2=0x7f2db40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 0x7f2db40b6120 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8104eb0 tx=0x7f2db000b390 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 msgr2=0x7f2dc8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2db800d950 tx=0x7f2db800de10 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 shutdown_connections 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2dc8108e60 0x7f2dc819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2db40b3c60 0x7f2db40b6120 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2dc81084e0 0x7f2dc81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 --2- 172.21.15.107:0/3338319114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2dc8102ad0 0x7f2dc8198e90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 >> 172.21.15.107:0/3338319114 conn(0x7f2dc80fe540 msgr2=0x7f2dc8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 shutdown_connections 2024-08-20T06:35:56.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.147+0000 7f2dcdb6f640 1 -- 172.21.15.107:0/3338319114 wait complete. 2024-08-20T06:35:56.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 -- 172.21.15.107:0/4122435373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 msgr2=0x7f15b410eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 --2- 172.21.15.107:0/4122435373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b410eba0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f159c009900 tx=0x7f159c02a410 comp rx=0 tx=0).stop 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 -- 172.21.15.107:0/4122435373 shutdown_connections 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 --2- 172.21.15.107:0/4122435373 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 0x7f15b41114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 --2- 172.21.15.107:0/4122435373 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b410eba0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 --2- 172.21.15.107:0/4122435373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 0x7f15b40755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 -- 172.21.15.107:0/4122435373 >> 172.21.15.107:0/4122435373 conn(0x7f15b40fe540 msgr2=0x7f15b4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 -- 172.21.15.107:0/4122435373 shutdown_connections 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.185+0000 7f15ba83f640 1 -- 172.21.15.107:0/4122435373 wait complete. 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 Processor -- start 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 -- start start 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 0x7f15b419b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:56.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 0x7f15b41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4114b70 con 0x7f15b4075b10 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15b41149f0 con 0x7f15b410f0e0 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15ba83f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15b4114cf0 con 0x7f15b40751f0 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15b37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15b37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60692/0 (socket says 172.21.15.107:60692) 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15b37fe640 1 -- 172.21.15.107:0/1953753033 learned_addr learned my addr 172.21.15.107:0/1953753033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15b3fff640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 0x7f15b419b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.186+0000 7f15b8ddc640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 0x7f15b41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 -- 172.21.15.107:0/1953753033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 msgr2=0x7f15b419b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 0x7f15b419b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 -- 172.21.15.107:0/1953753033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 msgr2=0x7f15b41a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 0x7f15b41a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 -- 172.21.15.107:0/1953753033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b41a1270 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b37fe640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f159c009230 tx=0x7f159c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c03f030 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f159c003a80 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b41a1500 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159c035070 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.187+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b41a7c20 con 0x7f15b4075b10 2024-08-20T06:35:56.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.188+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b40755d0 con 0x7f15b4075b10 2024-08-20T06:35:56.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.189+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f159c042030 con 0x7f15b4075b10 2024-08-20T06:35:56.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.190+0000 7f15b17fa640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 0x7f15a00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:56.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.190+0000 7f15b3fff640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 0x7f15a00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:56.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.190+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f159c104710 con 0x7f15b4075b10 2024-08-20T06:35:56.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.191+0000 7f15b3fff640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 0x7f15a00b6120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15a4005fd0 tx=0x7f15a400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:56.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.193+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f159c0cb6e0 con 0x7f15b4075b10 2024-08-20T06:35:56.289 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:56 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:35:56.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:56 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:35:56.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.374+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f15b419c2f0 con 0x7f15b4075b10 2024-08-20T06:35:56.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.375+0000 7f15b17fa640 1 -- 172.21.15.107:0/1953753033 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f159c0cfc30 con 0x7f15b4075b10 2024-08-20T06:35:56.376 INFO:teuthology.orchestra.run.smithi107.stdout:51539607622 2024-08-20T06:35:56.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.376+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 msgr2=0x7f15a00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 0x7f15a00b6120 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f15a4005fd0 tx=0x7f15a400b390 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 msgr2=0x7f15b419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f159c009230 tx=0x7f159c004270 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 shutdown_connections 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15b410f0e0 0x7f15b41a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15b4075b10 0x7f15b419b700 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15a00b3c60 0x7f15a00b6120 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 --2- 172.21.15.107:0/1953753033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b40751f0 0x7f15b419b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 >> 172.21.15.107:0/1953753033 conn(0x7f15b40fe540 msgr2=0x7f15b40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:56.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 shutdown_connections 2024-08-20T06:35:56.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.377+0000 7f15ba83f640 1 -- 172.21.15.107:0/1953753033 wait complete. 2024-08-20T06:35:56.403 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:56 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3338319114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1953753033' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/268420409' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:57 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3338319114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1953753033' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/268420409' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:57 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: pgmap v298: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:35:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3338319114' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T06:35:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1953753033' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T06:35:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/268420409' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T06:35:57.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:57 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:35:57.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 -- 172.21.15.107:0/1242706760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 msgr2=0x7f00bc10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 --2- 172.21.15.107:0/1242706760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc10f640 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009900 tx=0x7f00b402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 -- 172.21.15.107:0/1242706760 shutdown_connections 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 --2- 172.21.15.107:0/1242706760 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc10f640 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 --2- 172.21.15.107:0/1242706760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 0x7f00bc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 --2- 172.21.15.107:0/1242706760 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 -- 172.21.15.107:0/1242706760 >> 172.21.15.107:0/1242706760 conn(0x7f00bc0fe500 msgr2=0x7f00bc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 -- 172.21.15.107:0/1242706760 shutdown_connections 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.789+0000 7f00c3814640 1 -- 172.21.15.107:0/1242706760 wait complete. 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 Processor -- start 2024-08-20T06:35:57.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 -- start start 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 0x7f00bc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc1129b0 con 0x7f00bc108e40 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc112830 con 0x7f00bc102ab0 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c3814640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc112b30 con 0x7f00bc1084c0 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c1db1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c1db1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60752/0 (socket says 172.21.15.107:60752) 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c1db1640 1 -- 172.21.15.107:0/3257582078 learned_addr learned my addr 172.21.15.107:0/3257582078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.790+0000 7f00c15b0640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:57.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c0daf640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 0x7f00bc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 -- 172.21.15.107:0/3257582078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 msgr2=0x7f00bc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 0x7f00bc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 -- 172.21.15.107:0/3257582078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 msgr2=0x7f00bc0751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc0751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 -- 172.21.15.107:0/3257582078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc1a7240 con 0x7f00bc108e40 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c15b0640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c1db1640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00b4004770 tx=0x7f00b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b403f030 con 0x7f00bc108e40 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00b4004440 con 0x7f00bc108e40 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4035070 con 0x7f00bc108e40 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc1a74d0 con 0x7f00bc108e40 2024-08-20T06:35:57.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.791+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc1a79b0 con 0x7f00bc108e40 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.792+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc102e90 con 0x7f00bc108e40 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.793+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f00b4042030 con 0x7f00bc108e40 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.794+0000 7f00ba7fc640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 0x7f00a40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.794+0000 7f00c15b0640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 0x7f00a40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.794+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f00b4104000 con 0x7f00bc108e40 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.794+0000 7f00c15b0640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 0x7f00a40b6120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:57.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.798+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f00b40cb6a0 con 0x7f00bc108e40 2024-08-20T06:35:57.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.986+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f00bc10ac20 con 0x7f00bc108e40 2024-08-20T06:35:57.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.986+0000 7f00ba7fc640 1 -- 172.21.15.107:0/3257582078 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00b40cafa0 con 0x7f00bc108e40 2024-08-20T06:35:57.987 INFO:teuthology.orchestra.run.smithi107.stdout:73014444100 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 msgr2=0x7f00a40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 0x7f00a40b6120 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac009500 comp rx=0 tx=0).stop 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 msgr2=0x7f00bc075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00b4004770 tx=0x7f00b4004040 comp rx=0 tx=0).stop 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 shutdown_connections 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00bc108e40 0x7f00bc075c50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a40b3c60 0x7f00a40b6120 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00bc1084c0 0x7f00bc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 --2- 172.21.15.107:0/3257582078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00bc102ab0 0x7f00bc0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:57.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 >> 172.21.15.107:0/3257582078 conn(0x7f00bc0fe500 msgr2=0x7f00bc1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:57.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 shutdown_connections 2024-08-20T06:35:57.991 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.989+0000 7f00c3814640 1 -- 172.21.15.107:0/3257582078 wait complete. 2024-08-20T06:35:58.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2074305367' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T06:35:58.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3257582078' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T06:35:58.487 INFO:tasks.cephadm.ceph_manager.ceph:need seq 51539607619 got 51539607622 for osd.1 2024-08-20T06:35:58.487 DEBUG:teuthology.parallel:result is None 2024-08-20T06:35:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2074305367' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T06:35:58.488 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3257582078' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T06:35:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 -- 172.21.15.107:0/4270337007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb274101990 msgr2=0x7fb274101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 --2- 172.21.15.107:0/4270337007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb274101990 0x7fb274101d70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb264009900 tx=0x7fb26402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 -- 172.21.15.107:0/4270337007 shutdown_connections 2024-08-20T06:35:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 --2- 172.21.15.107:0/4270337007 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274107d20 0x7fb27410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 --2- 172.21.15.107:0/4270337007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2741073a0 0x7fb2741077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 --2- 172.21.15.107:0/4270337007 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb274101990 0x7fb274101d70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 -- 172.21.15.107:0/4270337007 >> 172.21.15.107:0/4270337007 conn(0x7fb2740fd400 msgr2=0x7fb2740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 -- 172.21.15.107:0/4270337007 shutdown_connections 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.405+0000 7fb278b50640 1 -- 172.21.15.107:0/4270337007 wait complete. 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.406+0000 7fb278b50640 1 Processor -- start 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.406+0000 7fb278b50640 1 -- start start 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274101990 0x7fb274199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb274107d20 0x7fb27419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb274111890 con 0x7fb2741073a0 2024-08-20T06:35:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb274111710 con 0x7fb274101990 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb274111a10 con 0x7fb274107d20 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60730/0 (socket says 172.21.15.107:60730) 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 -- 172.21.15.107:0/2074305367 learned_addr learned my addr 172.21.15.107:0/2074305367 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb27259c640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274101990 0x7fb274199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb272d9d640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb274107d20 0x7fb27419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 -- 172.21.15.107:0/2074305367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb274107d20 msgr2=0x7fb27419a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb274107d20 0x7fb27419a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 -- 172.21.15.107:0/2074305367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274101990 msgr2=0x7fb274199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274101990 0x7fb274199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 -- 172.21.15.107:0/2074305367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2741a5120 con 0x7fb2741073a0 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb269d9b640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00b4a0 tx=0x7fb25c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:58.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c014070 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb25c010aa0 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2741a5410 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c017b70 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.407+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2741a5980 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.408+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274101d70 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.410+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fb25c017070 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.410+0000 7fb26bfff640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 0x7fb2540b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.411+0000 7fb27259c640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 0x7fb2540b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.411+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7fb25c0e3260 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.411+0000 7fb27259c640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 0x7fb2540b6120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb2640057c0 tx=0x7fb2640058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.413+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb25c0aaa10 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.597+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fb27406abc0 con 0x7fb2741073a0 2024-08-20T06:35:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.597+0000 7fb26bfff640 1 -- 172.21.15.107:0/2074305367 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb25c0aa310 con 0x7fb2741073a0 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stdout:279172874264 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 msgr2=0x7fb2540b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 0x7fb2540b6120 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb2640057c0 tx=0x7fb2640058d0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 msgr2=0x7fb27419a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00b4a0 tx=0x7fb25c00b960 comp rx=0 tx=0).stop 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 shutdown_connections 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb2540b3c60 0x7fb2540b6120 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb274107d20 0x7fb27419a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb2741073a0 0x7fb27419a280 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 --2- 172.21.15.107:0/2074305367 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb274101990 0x7fb274199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 >> 172.21.15.107:0/2074305367 conn(0x7fb2740fd400 msgr2=0x7fb2740ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 shutdown_connections 2024-08-20T06:35:58.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:57.599+0000 7fb278b50640 1 -- 172.21.15.107:0/2074305367 wait complete. 2024-08-20T06:35:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2074305367' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T06:35:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3257582078' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T06:35:59.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 -- 172.21.15.107:0/445024146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef8101500 msgr2=0x7f7ef81018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:59.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 --2- 172.21.15.107:0/445024146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef8101500 0x7f7ef81018e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009900 tx=0x7f7ef402a410 comp rx=0 tx=0).stop 2024-08-20T06:35:59.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 -- 172.21.15.107:0/445024146 shutdown_connections 2024-08-20T06:35:59.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 --2- 172.21.15.107:0/445024146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef810cdc0 0x7f7ef810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 --2- 172.21.15.107:0/445024146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef810c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 --2- 172.21.15.107:0/445024146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef8101500 0x7f7ef81018e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 -- 172.21.15.107:0/445024146 >> 172.21.15.107:0/445024146 conn(0x7f7ef8076bd0 msgr2=0x7f7ef8076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 -- 172.21.15.107:0/445024146 shutdown_connections 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.124+0000 7f7efed16640 1 -- 172.21.15.107:0/445024146 wait complete. 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 Processor -- start 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 -- start start 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 0x7f7ef8198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef8199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112980 con 0x7f7ef810cdc0 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112800 con 0x7f7ef8101500 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efed16640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112b00 con 0x7f7ef8101f00 2024-08-20T06:35:59.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efd2b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efd2b3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60770/0 (socket says 172.21.15.107:60770) 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.125+0000 7f7efd2b3640 1 -- 172.21.15.107:0/3006542218 learned_addr learned my addr 172.21.15.107:0/3006542218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efcab2640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 0x7f7ef8198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7eeffff640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef8199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 -- 172.21.15.107:0/3006542218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 msgr2=0x7f7ef8199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef8199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 -- 172.21.15.107:0/3006542218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 msgr2=0x7f7ef8198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 0x7f7ef8198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 -- 172.21.15.107:0/3006542218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef819efe0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efcab2640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 0x7f7ef8198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:35:59.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7eeffff640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efd2b3640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000eaa0 tx=0x7f7ef000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef000cdb0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ef0005020 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef819f2d0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00116d0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.126+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef81a5a10 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.127+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef81018e0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.128+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f7ef00051d0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.129+0000 7f7eedffb640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 0x7f7ee00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.129+0000 7f7efcab2640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 0x7f7ee00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.129+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f7ef00e3e70 con 0x7f7ef810cdc0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.129+0000 7f7efcab2640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 0x7f7ee00b6120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:35:59.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.133+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7ef0016030 con 0x7f7ef810cdc0 2024-08-20T06:35:59.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.313+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7ef81077e0 con 0x7f7ef810cdc0 2024-08-20T06:35:59.315 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:35:59 smithi107 ceph-mon[18495]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:59.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.314+0000 7f7eedffb640 1 -- 172.21.15.107:0/3006542218 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7ef00ab510 con 0x7f7ef810cdc0 2024-08-20T06:35:59.315 INFO:teuthology.orchestra.run.smithi107.stdout:34359738444 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 msgr2=0x7f7ee00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 0x7f7ee00b6120 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4000c00 tx=0x7f7ef4005a20 comp rx=0 tx=0).stop 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 msgr2=0x7f7ef819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000eaa0 tx=0x7f7ef000ef60 comp rx=0 tx=0).stop 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 shutdown_connections 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7ef810cdc0 0x7f7ef819e810 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ee00b3c60 0x7f7ee00b6120 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ef8101f00 0x7f7ef8199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 --2- 172.21.15.107:0/3006542218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7ef8101500 0x7f7ef8198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:35:59.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 >> 172.21.15.107:0/3006542218 conn(0x7f7ef8076bd0 msgr2=0x7f7ef80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:35:59.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 shutdown_connections 2024-08-20T06:35:59.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:59.316+0000 7f7efed16640 1 -- 172.21.15.107:0/3006542218 wait complete. 2024-08-20T06:35:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:35:59 smithi183 ceph-mon[22204]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:35:59 smithi193 ceph-mon[21944]: pgmap v299: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:35:59.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168608 got 240518168609 for osd.9 2024-08-20T06:35:59.566 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:00.342 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:00 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3006542218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T06:36:00.371 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874261 got 279172874264 for osd.11 2024-08-20T06:36:00.371 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:00.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/2599909283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 msgr2=0x7f5bd4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:00.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/2599909283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4108920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009900 tx=0x7f5bbc02a410 comp rx=0 tx=0).stop 2024-08-20T06:36:00.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/2599909283 shutdown_connections 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/2599909283 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4108e60 0x7f5bd410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/2599909283 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4108920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/2599909283 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4102ad0 0x7f5bd4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/2599909283 >> 172.21.15.107:0/2599909283 conn(0x7f5bd40fe540 msgr2=0x7f5bd4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/2599909283 shutdown_connections 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.604+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/2599909283 wait complete. 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 Processor -- start 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 -- start start 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4102ad0 0x7f5bd4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4108e60 0x7f5bd419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd41129d0 con 0x7f5bd41084e0 2024-08-20T06:36:00.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4112850 con 0x7f5bd4108e60 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.605+0000 7f5bdaa5b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd4112b50 con 0x7f5bd4102ad0 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60712/0 (socket says 172.21.15.107:60712) 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 -- 172.21.15.107:0/268420409 learned_addr learned my addr 172.21.15.107:0/268420409 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bd3fff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4102ad0 0x7f5bd4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bd8ff8640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4108e60 0x7f5bd419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 -- 172.21.15.107:0/268420409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4102ad0 msgr2=0x7f5bd4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4102ad0 0x7f5bd4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 -- 172.21.15.107:0/268420409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4108e60 msgr2=0x7f5bd419e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4108e60 0x7f5bd419e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 -- 172.21.15.107:0/268420409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd419ef90 con 0x7f5bd41084e0 2024-08-20T06:36:00.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bcbfff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009230 tx=0x7f5bbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc03f030 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5bbc003a80 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd419f220 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc035070 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.606+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd41a5940 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.607+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd4102eb0 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.609+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f5bbc042030 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.609+0000 7f5bd1ffb640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 0x7f5bc00b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.609+0000 7f5bd3fff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 0x7f5bc00b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.609+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 7979+0+0 (secure 0 0 0) 0x7f5bbc104230 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.610+0000 7f5bd3fff640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 0x7f5bc00b61a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400bbe0 tx=0x7f5bc400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.613+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5bbc0cb280 con 0x7f5bd41084e0 2024-08-20T06:36:00.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.796+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5bd410ac40 con 0x7f5bd41084e0 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.796+0000 7f5bd1ffb640 1 -- 172.21.15.107:0/268420409 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5bbc0cf7d0 con 0x7f5bd41084e0 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stdout:219043332135 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.798+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 msgr2=0x7f5bc00b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 0x7f5bc00b61a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400bbe0 tx=0x7f5bc400b390 comp rx=0 tx=0).stop 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 msgr2=0x7f5bd4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc009230 tx=0x7f5bbc004270 comp rx=0 tx=0).stop 2024-08-20T06:36:00.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 shutdown_connections 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5bd4108e60 0x7f5bd419e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5bd41084e0 0x7f5bd4199420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5bc00b3ce0 0x7f5bc00b61a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 --2- 172.21.15.107:0/268420409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bd4102ad0 0x7f5bd4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 >> 172.21.15.107:0/268420409 conn(0x7f5bd40fe540 msgr2=0x7f5bd4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 shutdown_connections 2024-08-20T06:36:00.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:35:56.799+0000 7f5bdaa5b640 1 -- 172.21.15.107:0/268420409 wait complete. 2024-08-20T06:36:00.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:00 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3006542218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T06:36:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:00 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3006542218' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T06:36:01.100 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332133 got 219043332135 for osd.8 2024-08-20T06:36:01.100 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:01.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:01 smithi107 ceph-mon[18495]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:01.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:01 smithi183 ceph-mon[22204]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:01.519 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444097 got 73014444100 for osd.2 2024-08-20T06:36:01.519 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:01 smithi193 ceph-mon[21944]: pgmap v300: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:02.167 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738440 got 34359738444 for osd.0 2024-08-20T06:36:02.167 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]} : dispatch 2024-08-20T06:36:02.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:02 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 -- 172.21.15.107:0/1277132506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e41073a0 msgr2=0x7f20e41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 --2- 172.21.15.107:0/1277132506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e41073a0 0x7f20e41077e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009900 tx=0x7f20d002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 -- 172.21.15.107:0/1277132506 shutdown_connections 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 --2- 172.21.15.107:0/1277132506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e4107d20 0x7f20e410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 --2- 172.21.15.107:0/1277132506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e41073a0 0x7f20e41077e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 --2- 172.21.15.107:0/1277132506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 0x7f20e4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 -- 172.21.15.107:0/1277132506 >> 172.21.15.107:0/1277132506 conn(0x7f20e40fd400 msgr2=0x7f20e40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 -- 172.21.15.107:0/1277132506 shutdown_connections 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.368+0000 7f20eacc6640 1 -- 172.21.15.107:0/1277132506 wait complete. 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 Processor -- start 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 -- start start 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 0x7f20e4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e41073a0 0x7f20e4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111890 con 0x7f20e4107d20 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111710 con 0x7f20e41073a0 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20eacc6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111a10 con 0x7f20e4101990 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20e9263640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20e9263640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60784/0 (socket says 172.21.15.107:60784) 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20e9263640 1 -- 172.21.15.107:0/4136643825 learned_addr learned my addr 172.21.15.107:0/4136643825 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20e3fff640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e41073a0 0x7f20e4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.369+0000 7f20e8a62640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 0x7f20e4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 -- 172.21.15.107:0/4136643825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 msgr2=0x7f20e4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 0x7f20e4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 -- 172.21.15.107:0/4136643825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e41073a0 msgr2=0x7f20e4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e41073a0 0x7f20e4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 -- 172.21.15.107:0/4136643825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419df80 con 0x7f20e4107d20 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e9263640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00ebb0 tx=0x7f20dc009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:02.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc00e2e0 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419e210 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20dc007950 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc005680 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.370+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e406a7b0 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.371+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4102c40 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.372+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f20dc005820 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.373+0000 7f20e1ffb640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 0x7f20d40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.373+0000 7f20e8a62640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 0x7f20d40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.373+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8240+0+0 (secure 0 0 0) 0x7f20dc0e5280 con 0x7f20e4107d20 2024-08-20T06:36:02.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.374+0000 7f20e8a62640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 0x7f20d40b6120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009990 tx=0x7f20d8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:02.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.377+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f20dc0ac770 con 0x7f20e4107d20 2024-08-20T06:36:02.435 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]} : dispatch 2024-08-20T06:36:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:02 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]} : dispatch 2024-08-20T06:36:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-08-20T06:36:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]} : dispatch 2024-08-20T06:36:02.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:02 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T06:36:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.564+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f20e406b1d0 con 0x7f20e4107d20 2024-08-20T06:36:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.564+0000 7f20e1ffb640 1 -- 172.21.15.107:0/4136643825 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f20dc0ac070 con 0x7f20e4107d20 2024-08-20T06:36:02.565 INFO:teuthology.orchestra.run.smithi107.stdout:167503724592 2024-08-20T06:36:02.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 msgr2=0x7f20d40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 0x7f20d40b6120 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009990 tx=0x7f20d8009110 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 msgr2=0x7f20e419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f20dc00ebb0 tx=0x7f20dc009f60 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 shutdown_connections 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20e4107d20 0x7f20e419d7b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20e41073a0 0x7f20e4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20d40b3c60 0x7f20d40b6120 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 --2- 172.21.15.107:0/4136643825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20e4101990 0x7f20e4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 >> 172.21.15.107:0/4136643825 conn(0x7f20e40fd400 msgr2=0x7f20e40ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.566+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 shutdown_connections 2024-08-20T06:36:02.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.567+0000 7f20eacc6640 1 -- 172.21.15.107:0/4136643825 wait complete. 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 -- 172.21.15.107:0/3056372761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 msgr2=0x7fd048101d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 --2- 172.21.15.107:0/3056372761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048101d90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd038009900 tx=0x7fd03802a410 comp rx=0 tx=0).stop 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 -- 172.21.15.107:0/3056372761 shutdown_connections 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 --2- 172.21.15.107:0/3056372761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 --2- 172.21.15.107:0/3056372761 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 0x7fd048107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 --2- 172.21.15.107:0/3056372761 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048101d90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 -- 172.21.15.107:0/3056372761 >> 172.21.15.107:0/3056372761 conn(0x7fd0480fd420 msgr2=0x7fd0480ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 -- 172.21.15.107:0/3056372761 shutdown_connections 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.930+0000 7fd04e49a640 1 -- 172.21.15.107:0/3056372761 wait complete. 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 Processor -- start 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 -- start start 2024-08-20T06:36:02.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 0x7fd04819a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04819a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd0481118b0 con 0x7fd0481019b0 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd048111730 con 0x7fd0481073c0 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04e49a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd048111a30 con 0x7fd048107d40 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60800/0 (socket says 172.21.15.107:60800) 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 -- 172.21.15.107:0/1575916134 learned_addr learned my addr 172.21.15.107:0/1575916134 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd0477fe640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 0x7fd04819a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04ca37640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04819a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 -- 172.21.15.107:0/1575916134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 msgr2=0x7fd04819a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04819a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 -- 172.21.15.107:0/1575916134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 msgr2=0x7fd04819a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 0x7fd04819a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd047fff640 1 -- 172.21.15.107:0/1575916134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0481a50c0 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.931+0000 7fd04ca37640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd047fff640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd038009230 tx=0x7fd038004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd03803f030 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0481a5350 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd038003a80 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd038035070 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.932+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0481a5880 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.933+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd048101d90 con 0x7fd0481019b0 2024-08-20T06:36:02.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.934+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fd038042030 con 0x7fd0481019b0 2024-08-20T06:36:02.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.935+0000 7fd0457fa640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 0x7fd0300b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:02.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.935+0000 7fd0477fe640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 0x7fd0300b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:02.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.935+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 8240+0+0 (secure 0 0 0) 0x7fd038104510 con 0x7fd0481019b0 2024-08-20T06:36:02.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.935+0000 7fd0477fe640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 0x7fd0300b6120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd04806bd90 tx=0x7fd03c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:02.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:02.939+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd0380cba00 con 0x7fd0481019b0 2024-08-20T06:36:03.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.126+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fd04802e340 con 0x7fd0481019b0 2024-08-20T06:36:03.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.129+0000 7fd0457fa640 1 -- 172.21.15.107:0/1575916134 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0380cb300 con 0x7fd0481019b0 2024-08-20T06:36:03.130 INFO:teuthology.orchestra.run.smithi107.stdout:124554051643 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 msgr2=0x7fd0300b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 0x7fd0300b6120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd04806bd90 tx=0x7fd03c00b390 comp rx=0 tx=0).stop 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 msgr2=0x7fd048199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd038009230 tx=0x7fd038004270 comp rx=0 tx=0).stop 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 shutdown_connections 2024-08-20T06:36:03.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd0300b3c60 0x7fd0300b6120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd048107d40 0x7fd04819a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd0481073c0 0x7fd04819a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 --2- 172.21.15.107:0/1575916134 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd0481019b0 0x7fd048199ce0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.131+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 >> 172.21.15.107:0/1575916134 conn(0x7fd0480fd420 msgr2=0x7fd0480fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.132+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 shutdown_connections 2024-08-20T06:36:03.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:03.132+0000 7fd04e49a640 1 -- 172.21.15.107:0/1575916134 wait complete. 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T06:36:03.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T06:36:03.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:03 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4136643825' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T06:36:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-20T06:36:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-08-20T06:36:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-20T06:36:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T06:36:03.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:03 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4136643825' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T06:36:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: pgmap v301: 97 pgs: 97 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-08-20T06:36:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-08-20T06:36:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-08-20T06:36:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.c", "id": [3, 9]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 6]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 8]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T06:36:03.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:03 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4136643825' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T06:36:03.656 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724587 got 167503724592 for osd.6 2024-08-20T06:36:03.656 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:04.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:04 smithi107 ceph-mon[18495]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T06:36:04.238 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:04 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1575916134' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T06:36:04.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 124554051641 got 124554051643 for osd.4 2024-08-20T06:36:04.264 DEBUG:teuthology.parallel:result is None 2024-08-20T06:36:04.264 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T06:36:04.265 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:36:04.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:04 smithi183 ceph-mon[22204]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T06:36:04.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:04 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1575916134' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T06:36:04.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:04 smithi193 ceph-mon[21944]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T06:36:04.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:04 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1575916134' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T06:36:05.034 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:05.411 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:05 smithi107 ceph-mon[18495]: pgmap v304: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-20T06:36:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:05 smithi183 ceph-mon[22204]: pgmap v304: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-20T06:36:05.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 -- 172.21.15.107:0/1985011819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c4104300 msgr2=0x7f42c410c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:05.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 --2- 172.21.15.107:0/1985011819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c4104300 0x7f42c410c7f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b802a410 comp rx=0 tx=0).stop 2024-08-20T06:36:05.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 -- 172.21.15.107:0/1985011819 shutdown_connections 2024-08-20T06:36:05.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 --2- 172.21.15.107:0/1985011819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 0x7f42c410f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 --2- 172.21.15.107:0/1985011819 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c4104300 0x7f42c410c7f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 --2- 172.21.15.107:0/1985011819 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c41039e0 0x7f42c4103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 -- 172.21.15.107:0/1985011819 >> 172.21.15.107:0/1985011819 conn(0x7f42c40fd440 msgr2=0x7f42c40ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 -- 172.21.15.107:0/1985011819 shutdown_connections 2024-08-20T06:36:05.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.489+0000 7f42cae1d640 1 -- 172.21.15.107:0/1985011819 wait complete. 2024-08-20T06:36:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 Processor -- start 2024-08-20T06:36:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 -- start start 2024-08-20T06:36:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 0x7f42c419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:05.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 0x7f42c41a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4112960 con 0x7f42c41039e0 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41127e0 con 0x7f42c4104300 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42cae1d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4112ae0 con 0x7f42c410ce40 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42c8bb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42c8bb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54526/0 (socket says 172.21.15.107:54526) 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42c8bb9640 1 -- 172.21.15.107:0/3222064627 learned_addr learned my addr 172.21.15.107:0/3222064627 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42c3fff640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 0x7f42c419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.490+0000 7f42c93ba640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 0x7f42c41a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 -- 172.21.15.107:0/3222064627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 msgr2=0x7f42c41a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:05.494 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 0x7f42c41a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 -- 172.21.15.107:0/3222064627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 msgr2=0x7f42c419b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 0x7f42c419b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 -- 172.21.15.107:0/3222064627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c41a11b0 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c3fff640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 0x7f42c419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c8bb9640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42b400d950 tx=0x7f42b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4017030 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f42b4004020 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c41a14a0 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40055d0 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.491+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c41a7b60 con 0x7f42c41039e0 2024-08-20T06:36:05.495 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.492+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42c4103dc0 con 0x7f42c41039e0 2024-08-20T06:36:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.494+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f42b4009a30 con 0x7f42c41039e0 2024-08-20T06:36:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.494+0000 7f42c1ffb640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 0x7f42ac0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.494+0000 7f42c3fff640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 0x7f42ac0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:05.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.494+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f42b40e3910 con 0x7f42c41039e0 2024-08-20T06:36:05.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.495+0000 7f42c3fff640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 0x7f42ac0b6120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006010 tx=0x7f42b8005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:05.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.498+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f42b40aae00 con 0x7f42c41039e0 2024-08-20T06:36:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:05 smithi193 ceph-mon[21944]: pgmap v304: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 6 op/s 2024-08-20T06:36:05.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.695+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f42c4102d80 con 0x7f42ac0b3c60 2024-08-20T06:36:05.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.700+0000 7f42c1ffb640 1 -- 172.21.15.107:0/3222064627 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258676 (secure 0 0 0) 0x7f42c4102d80 con 0x7f42ac0b3c60 2024-08-20T06:36:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:05.706 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:05.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 msgr2=0x7f42ac0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 0x7f42ac0b6120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f42b8006010 tx=0x7f42b8005a20 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 msgr2=0x7f42c419b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42b400d950 tx=0x7f42b400de10 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 shutdown_connections 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f42ac0b3c60 0x7f42ac0b6120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42c410ce40 0x7f42c41a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f42c4104300 0x7f42c419b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 --2- 172.21.15.107:0/3222064627 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f42c41039e0 0x7f42c419b100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 >> 172.21.15.107:0/3222064627 conn(0x7f42c40fd440 msgr2=0x7f42c40ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 shutdown_connections 2024-08-20T06:36:05.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:05.704+0000 7f42cae1d640 1 -- 172.21.15.107:0/3222064627 wait complete. 2024-08-20T06:36:06.371 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-08-20T06:36:05.321023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":422,"num_read_kb":304,"num_write":286,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364604,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577220,"statfs":{"total":1151940427776,"available":1151567073280,"internally_reserved":0,"allocated":11440128,"data_stored":6398740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":19,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926320+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:35:00.926320+0000","last_peered":"2024-08-20T06:35:00.926320+0000","last_clean":"2024-08-20T06:35:00.926320+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:35:00.926320+0000","last_undegraded":"2024-08-20T06:35:00.926320+0000","last_fullsized":"2024-08-20T06:35:00.926320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123128+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:03.123128+0000","last_peered":"2024-08-20T06:36:03.123128+0000","last_clean":"2024-08-20T06:36:03.123128+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:03.123128+0000","last_undegraded":"2024-08-20T06:36:03.123128+0000","last_fullsized":"2024-08-20T06:36:03.123128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123174+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:03.123174+0000","last_peered":"2024-08-20T06:36:03.123174+0000","last_clean":"2024-08-20T06:36:03.123174+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:03.123174+0000","last_undegraded":"2024-08-20T06:36:03.123174+0000","last_fullsized":"2024-08-20T06:36:03.123174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918161+0000","last_change":"2024-08-20T06:34:59.933963+0000","last_active":"2024-08-20T06:35:00.918161+0000","last_peered":"2024-08-20T06:35:00.918161+0000","last_clean":"2024-08-20T06:35:00.918161+0000","last_became_active":"2024-08-20T06:34:59.933682+0000","last_became_peered":"2024-08-20T06:34:59.933682+0000","last_unstale":"2024-08-20T06:35:00.918161+0000","last_undegraded":"2024-08-20T06:35:00.918161+0000","last_fullsized":"2024-08-20T06:35:00.918161+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:07.550290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924480+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:35:00.924480+0000","last_peered":"2024-08-20T06:35:00.924480+0000","last_clean":"2024-08-20T06:35:00.924480+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:35:00.924480+0000","last_undegraded":"2024-08-20T06:35:00.924480+0000","last_fullsized":"2024-08-20T06:35:00.924480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127459+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:02.127459+0000","last_peered":"2024-08-20T06:36:02.127459+0000","last_clean":"2024-08-20T06:36:02.127459+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:02.127459+0000","last_undegraded":"2024-08-20T06:36:02.127459+0000","last_fullsized":"2024-08-20T06:36:02.127459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925543+0000","last_change":"2024-08-20T06:34:59.933789+0000","last_active":"2024-08-20T06:35:00.925543+0000","last_peered":"2024-08-20T06:35:00.925543+0000","last_clean":"2024-08-20T06:35:00.925543+0000","last_became_active":"2024-08-20T06:34:59.933461+0000","last_became_peered":"2024-08-20T06:34:59.933461+0000","last_unstale":"2024-08-20T06:35:00.925543+0000","last_undegraded":"2024-08-20T06:35:00.925543+0000","last_fullsized":"2024-08-20T06:35:00.925543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:31:37.924168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.124966+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:02.124966+0000","last_peered":"2024-08-20T06:36:02.124966+0000","last_clean":"2024-08-20T06:36:02.124966+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:02.124966+0000","last_undegraded":"2024-08-20T06:36:02.124966+0000","last_fullsized":"2024-08-20T06:36:02.124966+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129780+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:03.129780+0000","last_peered":"2024-08-20T06:36:03.129780+0000","last_clean":"2024-08-20T06:36:03.129780+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:03.129780+0000","last_undegraded":"2024-08-20T06:36:03.129780+0000","last_fullsized":"2024-08-20T06:36:03.129780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127840+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:02.127840+0000","last_peered":"2024-08-20T06:36:02.127840+0000","last_clean":"2024-08-20T06:36:02.127840+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:02.127840+0000","last_undegraded":"2024-08-20T06:36:02.127840+0000","last_fullsized":"2024-08-20T06:36:02.127840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926964+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:35:00.926964+0000","last_peered":"2024-08-20T06:35:00.926964+0000","last_clean":"2024-08-20T06:35:00.926964+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:35:00.926964+0000","last_undegraded":"2024-08-20T06:35:00.926964+0000","last_fullsized":"2024-08-20T06:35:00.926964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120512+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:02.120512+0000","last_peered":"2024-08-20T06:36:02.120512+0000","last_clean":"2024-08-20T06:36:02.120512+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:02.120512+0000","last_undegraded":"2024-08-20T06:36:02.120512+0000","last_fullsized":"2024-08-20T06:36:02.120512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120134+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:02.120134+0000","last_peered":"2024-08-20T06:36:02.120134+0000","last_clean":"2024-08-20T06:36:02.120134+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:02.120134+0000","last_undegraded":"2024-08-20T06:36:02.120134+0000","last_fullsized":"2024-08-20T06:36:02.120134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122863+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:03.122863+0000","last_peered":"2024-08-20T06:36:03.122863+0000","last_clean":"2024-08-20T06:36:03.122863+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:03.122863+0000","last_undegraded":"2024-08-20T06:36:03.122863+0000","last_fullsized":"2024-08-20T06:36:03.122863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122802+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:03.122802+0000","last_peered":"2024-08-20T06:36:03.122802+0000","last_clean":"2024-08-20T06:36:03.122802+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:03.122802+0000","last_undegraded":"2024-08-20T06:36:03.122802+0000","last_fullsized":"2024-08-20T06:36:03.122802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131241+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:03.131241+0000","last_peered":"2024-08-20T06:36:03.131241+0000","last_clean":"2024-08-20T06:36:03.131241+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:03.131241+0000","last_undegraded":"2024-08-20T06:36:03.131241+0000","last_fullsized":"2024-08-20T06:36:03.131241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926363+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:35:00.926363+0000","last_peered":"2024-08-20T06:35:00.926363+0000","last_clean":"2024-08-20T06:35:00.926363+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:35:00.926363+0000","last_undegraded":"2024-08-20T06:35:00.926363+0000","last_fullsized":"2024-08-20T06:35:00.926363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917047+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:35:00.917047+0000","last_peered":"2024-08-20T06:35:00.917047+0000","last_clean":"2024-08-20T06:35:00.917047+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:35:00.917047+0000","last_undegraded":"2024-08-20T06:35:00.917047+0000","last_fullsized":"2024-08-20T06:35:00.917047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122107+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:03.122107+0000","last_peered":"2024-08-20T06:36:03.122107+0000","last_clean":"2024-08-20T06:36:03.122107+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:03.122107+0000","last_undegraded":"2024-08-20T06:36:03.122107+0000","last_fullsized":"2024-08-20T06:36:03.122107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924758+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:35:00.924758+0000","last_peered":"2024-08-20T06:35:00.924758+0000","last_clean":"2024-08-20T06:35:00.924758+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:35:00.924758+0000","last_undegraded":"2024-08-20T06:35:00.924758+0000","last_fullsized":"2024-08-20T06:35:00.924758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120192+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:02.120192+0000","last_peered":"2024-08-20T06:36:02.120192+0000","last_clean":"2024-08-20T06:36:02.120192+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:02.120192+0000","last_undegraded":"2024-08-20T06:36:02.120192+0000","last_fullsized":"2024-08-20T06:36:02.120192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129402+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:03.129402+0000","last_peered":"2024-08-20T06:36:03.129402+0000","last_clean":"2024-08-20T06:36:03.129402+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:03.129402+0000","last_undegraded":"2024-08-20T06:36:03.129402+0000","last_fullsized":"2024-08-20T06:36:03.129402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125614+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:02.125614+0000","last_peered":"2024-08-20T06:36:02.125614+0000","last_clean":"2024-08-20T06:36:02.125614+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:02.125614+0000","last_undegraded":"2024-08-20T06:36:02.125614+0000","last_fullsized":"2024-08-20T06:36:02.125614+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120983+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:03.120983+0000","last_peered":"2024-08-20T06:36:03.120983+0000","last_clean":"2024-08-20T06:36:03.120983+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:03.120983+0000","last_undegraded":"2024-08-20T06:36:03.120983+0000","last_fullsized":"2024-08-20T06:36:03.120983+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125871+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:02.125871+0000","last_peered":"2024-08-20T06:36:02.125871+0000","last_clean":"2024-08-20T06:36:02.125871+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:02.125871+0000","last_undegraded":"2024-08-20T06:36:02.125871+0000","last_fullsized":"2024-08-20T06:36:02.125871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122256+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:03.122256+0000","last_peered":"2024-08-20T06:36:03.122256+0000","last_clean":"2024-08-20T06:36:03.122256+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:03.122256+0000","last_undegraded":"2024-08-20T06:36:03.122256+0000","last_fullsized":"2024-08-20T06:36:03.122256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925219+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:35:00.925219+0000","last_peered":"2024-08-20T06:35:00.925219+0000","last_clean":"2024-08-20T06:35:00.925219+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:35:00.925219+0000","last_undegraded":"2024-08-20T06:35:00.925219+0000","last_fullsized":"2024-08-20T06:35:00.925219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120039+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:02.120039+0000","last_peered":"2024-08-20T06:36:02.120039+0000","last_clean":"2024-08-20T06:36:02.120039+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:02.120039+0000","last_undegraded":"2024-08-20T06:36:02.120039+0000","last_fullsized":"2024-08-20T06:36:02.120039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123039+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:03.123039+0000","last_peered":"2024-08-20T06:36:03.123039+0000","last_clean":"2024-08-20T06:36:03.123039+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:03.123039+0000","last_undegraded":"2024-08-20T06:36:03.123039+0000","last_fullsized":"2024-08-20T06:36:03.123039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121952+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:03.121952+0000","last_peered":"2024-08-20T06:36:03.121952+0000","last_clean":"2024-08-20T06:36:03.121952+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:03.121952+0000","last_undegraded":"2024-08-20T06:36:03.121952+0000","last_fullsized":"2024-08-20T06:36:03.121952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122088+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:03.122088+0000","last_peered":"2024-08-20T06:36:03.122088+0000","last_clean":"2024-08-20T06:36:03.122088+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:03.122088+0000","last_undegraded":"2024-08-20T06:36:03.122088+0000","last_fullsized":"2024-08-20T06:36:03.122088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924729+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:35:00.924729+0000","last_peered":"2024-08-20T06:35:00.924729+0000","last_clean":"2024-08-20T06:35:00.924729+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:35:00.924729+0000","last_undegraded":"2024-08-20T06:35:00.924729+0000","last_fullsized":"2024-08-20T06:35:00.924729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924280+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:35:00.924280+0000","last_peered":"2024-08-20T06:35:00.924280+0000","last_clean":"2024-08-20T06:35:00.924280+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:35:00.924280+0000","last_undegraded":"2024-08-20T06:35:00.924280+0000","last_fullsized":"2024-08-20T06:35:00.924280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":86,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:04.065723+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:04.065723+0000","last_peered":"2024-08-20T06:36:04.065723+0000","last_clean":"2024-08-20T06:36:04.065723+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:04.065723+0000","last_undegraded":"2024-08-20T06:36:04.065723+0000","last_fullsized":"2024-08-20T06:36:04.065723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121272+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:03.121272+0000","last_peered":"2024-08-20T06:36:03.121272+0000","last_clean":"2024-08-20T06:36:03.121272+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:03.121272+0000","last_undegraded":"2024-08-20T06:36:03.121272+0000","last_fullsized":"2024-08-20T06:36:03.121272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":24,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127547+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:02.127547+0000","last_peered":"2024-08-20T06:36:02.127547+0000","last_clean":"2024-08-20T06:36:02.127547+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:02.127547+0000","last_undegraded":"2024-08-20T06:36:02.127547+0000","last_fullsized":"2024-08-20T06:36:02.127547+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119747+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:02.119747+0000","last_peered":"2024-08-20T06:36:02.119747+0000","last_clean":"2024-08-20T06:36:02.119747+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:02.119747+0000","last_undegraded":"2024-08-20T06:36:02.119747+0000","last_fullsized":"2024-08-20T06:36:02.119747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125036+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:02.125036+0000","last_peered":"2024-08-20T06:36:02.125036+0000","last_clean":"2024-08-20T06:36:02.125036+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:02.125036+0000","last_undegraded":"2024-08-20T06:36:02.125036+0000","last_fullsized":"2024-08-20T06:36:02.125036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916637+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:35:00.916637+0000","last_peered":"2024-08-20T06:35:00.916637+0000","last_clean":"2024-08-20T06:35:00.916637+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:35:00.916637+0000","last_undegraded":"2024-08-20T06:35:00.916637+0000","last_fullsized":"2024-08-20T06:35:00.916637+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":77,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127854+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:02.127854+0000","last_peered":"2024-08-20T06:36:02.127854+0000","last_clean":"2024-08-20T06:36:02.127854+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:02.127854+0000","last_undegraded":"2024-08-20T06:36:02.127854+0000","last_fullsized":"2024-08-20T06:36:02.127854+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129663+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:03.129663+0000","last_peered":"2024-08-20T06:36:03.129663+0000","last_clean":"2024-08-20T06:36:03.129663+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:03.129663+0000","last_undegraded":"2024-08-20T06:36:03.129663+0000","last_fullsized":"2024-08-20T06:36:03.129663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965368+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:35:07.965368+0000","last_peered":"2024-08-20T06:35:07.965368+0000","last_clean":"2024-08-20T06:35:07.965368+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:35:07.965368+0000","last_undegraded":"2024-08-20T06:35:07.965368+0000","last_fullsized":"2024-08-20T06:35:07.965368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.228019+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:03.228019+0000","last_peered":"2024-08-20T06:36:03.228019+0000","last_clean":"2024-08-20T06:36:03.228019+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:03.228019+0000","last_undegraded":"2024-08-20T06:36:03.228019+0000","last_fullsized":"2024-08-20T06:36:03.228019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129608+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:03.129608+0000","last_peered":"2024-08-20T06:36:03.129608+0000","last_clean":"2024-08-20T06:36:03.129608+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:03.129608+0000","last_undegraded":"2024-08-20T06:36:03.129608+0000","last_fullsized":"2024-08-20T06:36:03.129608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'29","reported_seq":163,"reported_epoch":71,"state":"peering","last_fresh":"2024-08-20T06:36:02.062278+0000","last_change":"2024-08-20T06:36:02.128052+0000","last_active":"2024-08-20T06:36:02.128052+0000","last_peered":"2024-08-20T06:36:02.062278+0000","last_clean":"2024-08-20T06:36:02.062278+0000","last_became_active":"2024-08-20T06:34:57.920116+0000","last_became_peered":"2024-08-20T06:34:57.920116+0000","last_unstale":"2024-08-20T06:36:02.062278+0000","last_undegraded":"2024-08-20T06:36:02.062278+0000","last_fullsized":"2024-08-20T06:36:02.062278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":116,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926259+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:35:00.926259+0000","last_peered":"2024-08-20T06:35:00.926259+0000","last_clean":"2024-08-20T06:35:00.926259+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:35:00.926259+0000","last_undegraded":"2024-08-20T06:35:00.926259+0000","last_fullsized":"2024-08-20T06:35:00.926259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972040+0000","last_change":"2024-08-20T06:34:57.919137+0000","last_active":"2024-08-20T06:35:08.972040+0000","last_peered":"2024-08-20T06:35:08.972040+0000","last_clean":"2024-08-20T06:35:08.972040+0000","last_became_active":"2024-08-20T06:34:57.918821+0000","last_became_peered":"2024-08-20T06:34:57.918821+0000","last_unstale":"2024-08-20T06:35:08.972040+0000","last_undegraded":"2024-08-20T06:35:08.972040+0000","last_fullsized":"2024-08-20T06:35:08.972040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:51:18.267609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119821+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:02.119821+0000","last_peered":"2024-08-20T06:36:02.119821+0000","last_clean":"2024-08-20T06:36:02.119821+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:02.119821+0000","last_undegraded":"2024-08-20T06:36:02.119821+0000","last_fullsized":"2024-08-20T06:36:02.119821+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925098+0000","last_change":"2024-08-20T06:34:59.930132+0000","last_active":"2024-08-20T06:35:00.925098+0000","last_peered":"2024-08-20T06:35:00.925098+0000","last_clean":"2024-08-20T06:35:00.925098+0000","last_became_active":"2024-08-20T06:34:59.929820+0000","last_became_peered":"2024-08-20T06:34:59.929820+0000","last_unstale":"2024-08-20T06:35:00.925098+0000","last_undegraded":"2024-08-20T06:35:00.925098+0000","last_fullsized":"2024-08-20T06:35:00.925098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:30.947526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":80,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120282+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:02.120282+0000","last_peered":"2024-08-20T06:36:02.120282+0000","last_clean":"2024-08-20T06:36:02.120282+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:02.120282+0000","last_undegraded":"2024-08-20T06:36:02.120282+0000","last_fullsized":"2024-08-20T06:36:02.120282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924673+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:35:00.924673+0000","last_peered":"2024-08-20T06:35:00.924673+0000","last_clean":"2024-08-20T06:35:00.924673+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:35:00.924673+0000","last_undegraded":"2024-08-20T06:35:00.924673+0000","last_fullsized":"2024-08-20T06:35:00.924673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973062+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:35:08.973062+0000","last_peered":"2024-08-20T06:35:08.973062+0000","last_clean":"2024-08-20T06:35:08.973062+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:35:08.973062+0000","last_undegraded":"2024-08-20T06:35:08.973062+0000","last_fullsized":"2024-08-20T06:35:08.973062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120911+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:03.120911+0000","last_peered":"2024-08-20T06:36:03.120911+0000","last_clean":"2024-08-20T06:36:03.120911+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:03.120911+0000","last_undegraded":"2024-08-20T06:36:03.120911+0000","last_fullsized":"2024-08-20T06:36:03.120911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980696+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:35:09.980696+0000","last_peered":"2024-08-20T06:35:09.980696+0000","last_clean":"2024-08-20T06:35:09.980696+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:35:09.980696+0000","last_undegraded":"2024-08-20T06:35:09.980696+0000","last_fullsized":"2024-08-20T06:35:09.980696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925884+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:35:00.925884+0000","last_peered":"2024-08-20T06:35:00.925884+0000","last_clean":"2024-08-20T06:35:00.925884+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:35:00.925884+0000","last_undegraded":"2024-08-20T06:35:00.925884+0000","last_fullsized":"2024-08-20T06:35:00.925884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120080+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:02.120080+0000","last_peered":"2024-08-20T06:36:02.120080+0000","last_clean":"2024-08-20T06:36:02.120080+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:02.120080+0000","last_undegraded":"2024-08-20T06:36:02.120080+0000","last_fullsized":"2024-08-20T06:36:02.120080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925856+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:35:00.925856+0000","last_peered":"2024-08-20T06:35:00.925856+0000","last_clean":"2024-08-20T06:35:00.925856+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:35:00.925856+0000","last_undegraded":"2024-08-20T06:35:00.925856+0000","last_fullsized":"2024-08-20T06:35:00.925856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.110553+0000","last_change":"2024-08-20T06:34:59.935512+0000","last_active":"2024-08-20T06:35:02.110553+0000","last_peered":"2024-08-20T06:35:02.110553+0000","last_clean":"2024-08-20T06:35:02.110553+0000","last_became_active":"2024-08-20T06:34:59.933714+0000","last_became_peered":"2024-08-20T06:34:59.933714+0000","last_unstale":"2024-08-20T06:35:02.110553+0000","last_undegraded":"2024-08-20T06:35:02.110553+0000","last_fullsized":"2024-08-20T06:35:02.110553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:55:50.042593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927132+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:35:00.927132+0000","last_peered":"2024-08-20T06:35:00.927132+0000","last_clean":"2024-08-20T06:35:00.927132+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:35:00.927132+0000","last_undegraded":"2024-08-20T06:35:00.927132+0000","last_fullsized":"2024-08-20T06:35:00.927132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924980+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:35:00.924980+0000","last_peered":"2024-08-20T06:35:00.924980+0000","last_clean":"2024-08-20T06:35:00.924980+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:35:00.924980+0000","last_undegraded":"2024-08-20T06:35:00.924980+0000","last_fullsized":"2024-08-20T06:35:00.924980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":26,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120466+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:02.120466+0000","last_peered":"2024-08-20T06:36:02.120466+0000","last_clean":"2024-08-20T06:36:02.120466+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:02.120466+0000","last_undegraded":"2024-08-20T06:36:02.120466+0000","last_fullsized":"2024-08-20T06:36:02.120466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125735+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:02.125735+0000","last_peered":"2024-08-20T06:36:02.125735+0000","last_clean":"2024-08-20T06:36:02.125735+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:02.125735+0000","last_undegraded":"2024-08-20T06:36:02.125735+0000","last_fullsized":"2024-08-20T06:36:02.125735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972913+0000","last_change":"2024-08-20T06:34:57.914079+0000","last_active":"2024-08-20T06:35:08.972913+0000","last_peered":"2024-08-20T06:35:08.972913+0000","last_clean":"2024-08-20T06:35:08.972913+0000","last_became_active":"2024-08-20T06:34:57.913747+0000","last_became_peered":"2024-08-20T06:34:57.913747+0000","last_unstale":"2024-08-20T06:35:08.972913+0000","last_undegraded":"2024-08-20T06:35:08.972913+0000","last_fullsized":"2024-08-20T06:35:08.972913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:21:24.138487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119732+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:02.119732+0000","last_peered":"2024-08-20T06:36:02.119732+0000","last_clean":"2024-08-20T06:36:02.119732+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:02.119732+0000","last_undegraded":"2024-08-20T06:36:02.119732+0000","last_fullsized":"2024-08-20T06:36:02.119732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":78,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119839+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:02.119839+0000","last_peered":"2024-08-20T06:36:02.119839+0000","last_clean":"2024-08-20T06:36:02.119839+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:02.119839+0000","last_undegraded":"2024-08-20T06:36:02.119839+0000","last_fullsized":"2024-08-20T06:36:02.119839+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120920+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:03.120920+0000","last_peered":"2024-08-20T06:36:03.120920+0000","last_clean":"2024-08-20T06:36:03.120920+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:03.120920+0000","last_undegraded":"2024-08-20T06:36:03.120920+0000","last_fullsized":"2024-08-20T06:36:03.120920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925692+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:35:00.925692+0000","last_peered":"2024-08-20T06:35:00.925692+0000","last_clean":"2024-08-20T06:35:00.925692+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:35:00.925692+0000","last_undegraded":"2024-08-20T06:35:00.925692+0000","last_fullsized":"2024-08-20T06:35:00.925692+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127664+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:02.127664+0000","last_peered":"2024-08-20T06:36:02.127664+0000","last_clean":"2024-08-20T06:36:02.127664+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:02.127664+0000","last_undegraded":"2024-08-20T06:36:02.127664+0000","last_fullsized":"2024-08-20T06:36:02.127664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916869+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:35:00.916869+0000","last_peered":"2024-08-20T06:35:00.916869+0000","last_clean":"2024-08-20T06:35:00.916869+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:35:00.916869+0000","last_undegraded":"2024-08-20T06:35:00.916869+0000","last_fullsized":"2024-08-20T06:35:00.916869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127733+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:02.127733+0000","last_peered":"2024-08-20T06:36:02.127733+0000","last_clean":"2024-08-20T06:36:02.127733+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:02.127733+0000","last_undegraded":"2024-08-20T06:36:02.127733+0000","last_fullsized":"2024-08-20T06:36:02.127733+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965120+0000","last_change":"2024-08-20T06:34:57.973505+0000","last_active":"2024-08-20T06:35:07.965120+0000","last_peered":"2024-08-20T06:35:07.965120+0000","last_clean":"2024-08-20T06:35:07.965120+0000","last_became_active":"2024-08-20T06:34:57.973120+0000","last_became_peered":"2024-08-20T06:34:57.973120+0000","last_unstale":"2024-08-20T06:35:07.965120+0000","last_undegraded":"2024-08-20T06:35:07.965120+0000","last_fullsized":"2024-08-20T06:35:07.965120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:29:02.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120065+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:02.120065+0000","last_peered":"2024-08-20T06:36:02.120065+0000","last_clean":"2024-08-20T06:36:02.120065+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:02.120065+0000","last_undegraded":"2024-08-20T06:36:02.120065+0000","last_fullsized":"2024-08-20T06:36:02.120065+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'10","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:23.924664+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:35:23.924664+0000","last_peered":"2024-08-20T06:35:23.924664+0000","last_clean":"2024-08-20T06:35:23.924664+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:35:23.924664+0000","last_undegraded":"2024-08-20T06:35:23.924664+0000","last_fullsized":"2024-08-20T06:35:23.924664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125987+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:02.125987+0000","last_peered":"2024-08-20T06:36:02.125987+0000","last_clean":"2024-08-20T06:36:02.125987+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:02.125987+0000","last_undegraded":"2024-08-20T06:36:02.125987+0000","last_fullsized":"2024-08-20T06:36:02.125987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120577+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:02.120577+0000","last_peered":"2024-08-20T06:36:02.120577+0000","last_clean":"2024-08-20T06:36:02.120577+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:02.120577+0000","last_undegraded":"2024-08-20T06:36:02.120577+0000","last_fullsized":"2024-08-20T06:36:02.120577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123213+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:03.123213+0000","last_peered":"2024-08-20T06:36:03.123213+0000","last_clean":"2024-08-20T06:36:03.123213+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:03.123213+0000","last_undegraded":"2024-08-20T06:36:03.123213+0000","last_fullsized":"2024-08-20T06:36:03.123213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924239+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:35:00.924239+0000","last_peered":"2024-08-20T06:35:00.924239+0000","last_clean":"2024-08-20T06:35:00.924239+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:35:00.924239+0000","last_undegraded":"2024-08-20T06:35:00.924239+0000","last_fullsized":"2024-08-20T06:35:00.924239+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":24,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125720+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:02.125720+0000","last_peered":"2024-08-20T06:36:02.125720+0000","last_clean":"2024-08-20T06:36:02.125720+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:02.125720+0000","last_undegraded":"2024-08-20T06:36:02.125720+0000","last_fullsized":"2024-08-20T06:36:02.125720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924929+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:35:00.924929+0000","last_peered":"2024-08-20T06:35:00.924929+0000","last_clean":"2024-08-20T06:35:00.924929+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:35:00.924929+0000","last_undegraded":"2024-08-20T06:35:00.924929+0000","last_fullsized":"2024-08-20T06:35:00.924929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125796+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:02.125796+0000","last_peered":"2024-08-20T06:36:02.125796+0000","last_clean":"2024-08-20T06:36:02.125796+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:02.125796+0000","last_undegraded":"2024-08-20T06:36:02.125796+0000","last_fullsized":"2024-08-20T06:36:02.125796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965620+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:35:07.965620+0000","last_peered":"2024-08-20T06:35:07.965620+0000","last_clean":"2024-08-20T06:35:07.965620+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:35:07.965620+0000","last_undegraded":"2024-08-20T06:35:07.965620+0000","last_fullsized":"2024-08-20T06:35:07.965620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928848+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:35:00.928848+0000","last_peered":"2024-08-20T06:35:00.928848+0000","last_clean":"2024-08-20T06:35:00.928848+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:35:00.928848+0000","last_undegraded":"2024-08-20T06:35:00.928848+0000","last_fullsized":"2024-08-20T06:35:00.928848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121464+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:03.121464+0000","last_peered":"2024-08-20T06:36:03.121464+0000","last_clean":"2024-08-20T06:36:03.121464+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:03.121464+0000","last_undegraded":"2024-08-20T06:36:03.121464+0000","last_fullsized":"2024-08-20T06:36:03.121464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930358+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:35:00.930358+0000","last_peered":"2024-08-20T06:35:00.930358+0000","last_clean":"2024-08-20T06:35:00.930358+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:35:00.930358+0000","last_undegraded":"2024-08-20T06:35:00.930358+0000","last_fullsized":"2024-08-20T06:35:00.930358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129497+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:03.129497+0000","last_peered":"2024-08-20T06:36:03.129497+0000","last_clean":"2024-08-20T06:36:03.129497+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:03.129497+0000","last_undegraded":"2024-08-20T06:36:03.129497+0000","last_fullsized":"2024-08-20T06:36:03.129497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971790+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:35:08.971790+0000","last_peered":"2024-08-20T06:35:08.971790+0000","last_clean":"2024-08-20T06:35:08.971790+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:35:08.971790+0000","last_undegraded":"2024-08-20T06:35:08.971790+0000","last_fullsized":"2024-08-20T06:35:08.971790+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123408+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:03.123408+0000","last_peered":"2024-08-20T06:36:03.123408+0000","last_clean":"2024-08-20T06:36:03.123408+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:03.123408+0000","last_undegraded":"2024-08-20T06:36:03.123408+0000","last_fullsized":"2024-08-20T06:36:03.123408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.126171+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:02.126171+0000","last_peered":"2024-08-20T06:36:02.126171+0000","last_clean":"2024-08-20T06:36:02.126171+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:02.126171+0000","last_undegraded":"2024-08-20T06:36:02.126171+0000","last_fullsized":"2024-08-20T06:36:02.126171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125991+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:02.125991+0000","last_peered":"2024-08-20T06:36:02.125991+0000","last_clean":"2024-08-20T06:36:02.125991+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:02.125991+0000","last_undegraded":"2024-08-20T06:36:02.125991+0000","last_fullsized":"2024-08-20T06:36:02.125991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973139+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:35:08.973139+0000","last_peered":"2024-08-20T06:35:08.973139+0000","last_clean":"2024-08-20T06:35:08.973139+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:35:08.973139+0000","last_undegraded":"2024-08-20T06:35:08.973139+0000","last_fullsized":"2024-08-20T06:35:08.973139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":346,"num_read_kb":240,"num_write":161,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":134296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717380,"statfs":{"total":95995035648,"available":95966597120,"internally_reserved":0,"allocated":765952,"data_stored":368498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":1021729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332137,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528365,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1376256,"data_stored":960193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":376824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1421312,"data_stored":977058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":790528,"data_stored":377261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":368434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607624,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":446847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":806912,"data_stored":378095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:06.376 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:36:06.637 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:07.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- 172.21.15.107:0/4002984138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8107d20 msgr2=0x7fd1a810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:07.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 --2- 172.21.15.107:0/4002984138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8107d20 0x7fd1a810e520 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0009900 tx=0x7fd1a002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:07.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- 172.21.15.107:0/4002984138 shutdown_connections 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 --2- 172.21.15.107:0/4002984138 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8107d20 0x7fd1a810e520 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 --2- 172.21.15.107:0/4002984138 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a81073a0 0x7fd1a81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 --2- 172.21.15.107:0/4002984138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1a8101990 0x7fd1a8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- 172.21.15.107:0/4002984138 >> 172.21.15.107:0/4002984138 conn(0x7fd1a80fd400 msgr2=0x7fd1a80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- 172.21.15.107:0/4002984138 shutdown_connections 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- 172.21.15.107:0/4002984138 wait complete. 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 Processor -- start 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.106+0000 7fd1b0155640 1 -- start start 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1a81073a0 0x7fd1a810f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8107d20 0x7fd1a810fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a81129d0 con 0x7fd1a8101990 2024-08-20T06:36:07.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112850 con 0x7fd1a8107d20 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a8112b50 con 0x7fd1a81073a0 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54532/0 (socket says 172.21.15.107:54532) 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 -- 172.21.15.107:0/2770801037 learned_addr learned my addr 172.21.15.107:0/2770801037 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 -- 172.21.15.107:0/2770801037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1a81073a0 msgr2=0x7fd1a810f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1a81073a0 0x7fd1a810f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 -- 172.21.15.107:0/2770801037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8107d20 msgr2=0x7fd1a810fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8107d20 0x7fd1a810fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 -- 172.21.15.107:0/2770801037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a81a7440 con 0x7fd1a8101990 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1adef1640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd19800ba50 tx=0x7fd19800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198019070 con 0x7fd1a8101990 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a81a7730 con 0x7fd1a8101990 2024-08-20T06:36:07.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd19800cbe0 con 0x7fd1a8101990 2024-08-20T06:36:07.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1980075d0 con 0x7fd1a8101990 2024-08-20T06:36:07.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.107+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a81a7c70 con 0x7fd1a8101990 2024-08-20T06:36:07.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.108+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a8101df0 con 0x7fd1a8101990 2024-08-20T06:36:07.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.110+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fd19801c070 con 0x7fd1a8101990 2024-08-20T06:36:07.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.111+0000 7fd1a6ffd640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 0x7fd1900b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:07.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.111+0000 7fd1ad6f0640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 0x7fd1900b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:07.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.112+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd1980e3bd0 con 0x7fd1a8101990 2024-08-20T06:36:07.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.112+0000 7fd1ad6f0640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 0x7fd1900b61a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8110b20 tx=0x7fd19c006c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:07.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.114+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd198023070 con 0x7fd1a8101990 2024-08-20T06:36:07.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.290+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1a80632c0 con 0x7fd1900b3ce0 2024-08-20T06:36:07.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: pgmap v305: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-20T06:36:07.291 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:07.292 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.292 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.292 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:36:07.292 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:36:07.292 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.295+0000 7fd1a6ffd640 1 -- 172.21.15.107:0/2770801037 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258676 (secure 0 0 0) 0x7fd1a80632c0 con 0x7fd1900b3ce0 2024-08-20T06:36:07.298 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:07.299 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 msgr2=0x7fd1900b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 0x7fd1900b61a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8110b20 tx=0x7fd19c006c90 comp rx=0 tx=0).stop 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 msgr2=0x7fd1a810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd19800ba50 tx=0x7fd19800bf10 comp rx=0 tx=0).stop 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 shutdown_connections 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd1a8107d20 0x7fd1a810fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd1900b3ce0 0x7fd1900b61a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1a81073a0 0x7fd1a810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 --2- 172.21.15.107:0/2770801037 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1a8101990 0x7fd1a810f030 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 >> 172.21.15.107:0/2770801037 conn(0x7fd1a80fd400 msgr2=0x7fd1a80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 shutdown_connections 2024-08-20T06:36:07.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:07.298+0000 7fd1b0155640 1 -- 172.21.15.107:0/2770801037 wait complete. 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: pgmap v305: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:36:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: pgmap v305: 97 pgs: 1 peering, 96 active+clean; 641 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-08-20T06:36:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='client.15087 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:36:07.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:36:07.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:36:07.975 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-08-20T06:36:05.321023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":422,"num_read_kb":304,"num_write":286,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364604,"kb_used_data":11172,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577220,"statfs":{"total":1151940427776,"available":1151567073280,"internally_reserved":0,"allocated":11440128,"data_stored":6398740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":43,"num_read_kb":19,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001697"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926320+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:35:00.926320+0000","last_peered":"2024-08-20T06:35:00.926320+0000","last_clean":"2024-08-20T06:35:00.926320+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:35:00.926320+0000","last_undegraded":"2024-08-20T06:35:00.926320+0000","last_fullsized":"2024-08-20T06:35:00.926320+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123128+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:03.123128+0000","last_peered":"2024-08-20T06:36:03.123128+0000","last_clean":"2024-08-20T06:36:03.123128+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:03.123128+0000","last_undegraded":"2024-08-20T06:36:03.123128+0000","last_fullsized":"2024-08-20T06:36:03.123128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123174+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:03.123174+0000","last_peered":"2024-08-20T06:36:03.123174+0000","last_clean":"2024-08-20T06:36:03.123174+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:03.123174+0000","last_undegraded":"2024-08-20T06:36:03.123174+0000","last_fullsized":"2024-08-20T06:36:03.123174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.918161+0000","last_change":"2024-08-20T06:34:59.933963+0000","last_active":"2024-08-20T06:35:00.918161+0000","last_peered":"2024-08-20T06:35:00.918161+0000","last_clean":"2024-08-20T06:35:00.918161+0000","last_became_active":"2024-08-20T06:34:59.933682+0000","last_became_peered":"2024-08-20T06:34:59.933682+0000","last_unstale":"2024-08-20T06:35:00.918161+0000","last_undegraded":"2024-08-20T06:35:00.918161+0000","last_fullsized":"2024-08-20T06:35:00.918161+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:07.550290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924480+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:35:00.924480+0000","last_peered":"2024-08-20T06:35:00.924480+0000","last_clean":"2024-08-20T06:35:00.924480+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:35:00.924480+0000","last_undegraded":"2024-08-20T06:35:00.924480+0000","last_fullsized":"2024-08-20T06:35:00.924480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127459+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:02.127459+0000","last_peered":"2024-08-20T06:36:02.127459+0000","last_clean":"2024-08-20T06:36:02.127459+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:02.127459+0000","last_undegraded":"2024-08-20T06:36:02.127459+0000","last_fullsized":"2024-08-20T06:36:02.127459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925543+0000","last_change":"2024-08-20T06:34:59.933789+0000","last_active":"2024-08-20T06:35:00.925543+0000","last_peered":"2024-08-20T06:35:00.925543+0000","last_clean":"2024-08-20T06:35:00.925543+0000","last_became_active":"2024-08-20T06:34:59.933461+0000","last_became_peered":"2024-08-20T06:34:59.933461+0000","last_unstale":"2024-08-20T06:35:00.925543+0000","last_undegraded":"2024-08-20T06:35:00.925543+0000","last_fullsized":"2024-08-20T06:35:00.925543+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:31:37.924168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.124966+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:02.124966+0000","last_peered":"2024-08-20T06:36:02.124966+0000","last_clean":"2024-08-20T06:36:02.124966+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:02.124966+0000","last_undegraded":"2024-08-20T06:36:02.124966+0000","last_fullsized":"2024-08-20T06:36:02.124966+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129780+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:03.129780+0000","last_peered":"2024-08-20T06:36:03.129780+0000","last_clean":"2024-08-20T06:36:03.129780+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:03.129780+0000","last_undegraded":"2024-08-20T06:36:03.129780+0000","last_fullsized":"2024-08-20T06:36:03.129780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127840+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:02.127840+0000","last_peered":"2024-08-20T06:36:02.127840+0000","last_clean":"2024-08-20T06:36:02.127840+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:02.127840+0000","last_undegraded":"2024-08-20T06:36:02.127840+0000","last_fullsized":"2024-08-20T06:36:02.127840+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926964+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:35:00.926964+0000","last_peered":"2024-08-20T06:35:00.926964+0000","last_clean":"2024-08-20T06:35:00.926964+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:35:00.926964+0000","last_undegraded":"2024-08-20T06:35:00.926964+0000","last_fullsized":"2024-08-20T06:35:00.926964+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120512+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:02.120512+0000","last_peered":"2024-08-20T06:36:02.120512+0000","last_clean":"2024-08-20T06:36:02.120512+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:02.120512+0000","last_undegraded":"2024-08-20T06:36:02.120512+0000","last_fullsized":"2024-08-20T06:36:02.120512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120134+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:02.120134+0000","last_peered":"2024-08-20T06:36:02.120134+0000","last_clean":"2024-08-20T06:36:02.120134+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:02.120134+0000","last_undegraded":"2024-08-20T06:36:02.120134+0000","last_fullsized":"2024-08-20T06:36:02.120134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122863+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:03.122863+0000","last_peered":"2024-08-20T06:36:03.122863+0000","last_clean":"2024-08-20T06:36:03.122863+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:03.122863+0000","last_undegraded":"2024-08-20T06:36:03.122863+0000","last_fullsized":"2024-08-20T06:36:03.122863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122802+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:03.122802+0000","last_peered":"2024-08-20T06:36:03.122802+0000","last_clean":"2024-08-20T06:36:03.122802+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:03.122802+0000","last_undegraded":"2024-08-20T06:36:03.122802+0000","last_fullsized":"2024-08-20T06:36:03.122802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131241+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:03.131241+0000","last_peered":"2024-08-20T06:36:03.131241+0000","last_clean":"2024-08-20T06:36:03.131241+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:03.131241+0000","last_undegraded":"2024-08-20T06:36:03.131241+0000","last_fullsized":"2024-08-20T06:36:03.131241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926363+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:35:00.926363+0000","last_peered":"2024-08-20T06:35:00.926363+0000","last_clean":"2024-08-20T06:35:00.926363+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:35:00.926363+0000","last_undegraded":"2024-08-20T06:35:00.926363+0000","last_fullsized":"2024-08-20T06:35:00.926363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.917047+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:35:00.917047+0000","last_peered":"2024-08-20T06:35:00.917047+0000","last_clean":"2024-08-20T06:35:00.917047+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:35:00.917047+0000","last_undegraded":"2024-08-20T06:35:00.917047+0000","last_fullsized":"2024-08-20T06:35:00.917047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122107+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:03.122107+0000","last_peered":"2024-08-20T06:36:03.122107+0000","last_clean":"2024-08-20T06:36:03.122107+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:03.122107+0000","last_undegraded":"2024-08-20T06:36:03.122107+0000","last_fullsized":"2024-08-20T06:36:03.122107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924758+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:35:00.924758+0000","last_peered":"2024-08-20T06:35:00.924758+0000","last_clean":"2024-08-20T06:35:00.924758+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:35:00.924758+0000","last_undegraded":"2024-08-20T06:35:00.924758+0000","last_fullsized":"2024-08-20T06:35:00.924758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120192+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:02.120192+0000","last_peered":"2024-08-20T06:36:02.120192+0000","last_clean":"2024-08-20T06:36:02.120192+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:02.120192+0000","last_undegraded":"2024-08-20T06:36:02.120192+0000","last_fullsized":"2024-08-20T06:36:02.120192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129402+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:03.129402+0000","last_peered":"2024-08-20T06:36:03.129402+0000","last_clean":"2024-08-20T06:36:03.129402+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:03.129402+0000","last_undegraded":"2024-08-20T06:36:03.129402+0000","last_fullsized":"2024-08-20T06:36:03.129402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125614+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:02.125614+0000","last_peered":"2024-08-20T06:36:02.125614+0000","last_clean":"2024-08-20T06:36:02.125614+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:02.125614+0000","last_undegraded":"2024-08-20T06:36:02.125614+0000","last_fullsized":"2024-08-20T06:36:02.125614+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120983+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:03.120983+0000","last_peered":"2024-08-20T06:36:03.120983+0000","last_clean":"2024-08-20T06:36:03.120983+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:03.120983+0000","last_undegraded":"2024-08-20T06:36:03.120983+0000","last_fullsized":"2024-08-20T06:36:03.120983+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125871+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:02.125871+0000","last_peered":"2024-08-20T06:36:02.125871+0000","last_clean":"2024-08-20T06:36:02.125871+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:02.125871+0000","last_undegraded":"2024-08-20T06:36:02.125871+0000","last_fullsized":"2024-08-20T06:36:02.125871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122256+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:03.122256+0000","last_peered":"2024-08-20T06:36:03.122256+0000","last_clean":"2024-08-20T06:36:03.122256+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:03.122256+0000","last_undegraded":"2024-08-20T06:36:03.122256+0000","last_fullsized":"2024-08-20T06:36:03.122256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925219+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:35:00.925219+0000","last_peered":"2024-08-20T06:35:00.925219+0000","last_clean":"2024-08-20T06:35:00.925219+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:35:00.925219+0000","last_undegraded":"2024-08-20T06:35:00.925219+0000","last_fullsized":"2024-08-20T06:35:00.925219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120039+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:02.120039+0000","last_peered":"2024-08-20T06:36:02.120039+0000","last_clean":"2024-08-20T06:36:02.120039+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:02.120039+0000","last_undegraded":"2024-08-20T06:36:02.120039+0000","last_fullsized":"2024-08-20T06:36:02.120039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123039+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:03.123039+0000","last_peered":"2024-08-20T06:36:03.123039+0000","last_clean":"2024-08-20T06:36:03.123039+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:03.123039+0000","last_undegraded":"2024-08-20T06:36:03.123039+0000","last_fullsized":"2024-08-20T06:36:03.123039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121952+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:03.121952+0000","last_peered":"2024-08-20T06:36:03.121952+0000","last_clean":"2024-08-20T06:36:03.121952+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:03.121952+0000","last_undegraded":"2024-08-20T06:36:03.121952+0000","last_fullsized":"2024-08-20T06:36:03.121952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122088+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:03.122088+0000","last_peered":"2024-08-20T06:36:03.122088+0000","last_clean":"2024-08-20T06:36:03.122088+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:03.122088+0000","last_undegraded":"2024-08-20T06:36:03.122088+0000","last_fullsized":"2024-08-20T06:36:03.122088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924729+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:35:00.924729+0000","last_peered":"2024-08-20T06:35:00.924729+0000","last_clean":"2024-08-20T06:35:00.924729+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:35:00.924729+0000","last_undegraded":"2024-08-20T06:35:00.924729+0000","last_fullsized":"2024-08-20T06:35:00.924729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924280+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:35:00.924280+0000","last_peered":"2024-08-20T06:35:00.924280+0000","last_clean":"2024-08-20T06:35:00.924280+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:35:00.924280+0000","last_undegraded":"2024-08-20T06:35:00.924280+0000","last_fullsized":"2024-08-20T06:35:00.924280+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":86,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:04.065723+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:04.065723+0000","last_peered":"2024-08-20T06:36:04.065723+0000","last_clean":"2024-08-20T06:36:04.065723+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:04.065723+0000","last_undegraded":"2024-08-20T06:36:04.065723+0000","last_fullsized":"2024-08-20T06:36:04.065723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121272+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:03.121272+0000","last_peered":"2024-08-20T06:36:03.121272+0000","last_clean":"2024-08-20T06:36:03.121272+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:03.121272+0000","last_undegraded":"2024-08-20T06:36:03.121272+0000","last_fullsized":"2024-08-20T06:36:03.121272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":24,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127547+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:02.127547+0000","last_peered":"2024-08-20T06:36:02.127547+0000","last_clean":"2024-08-20T06:36:02.127547+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:02.127547+0000","last_undegraded":"2024-08-20T06:36:02.127547+0000","last_fullsized":"2024-08-20T06:36:02.127547+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119747+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:02.119747+0000","last_peered":"2024-08-20T06:36:02.119747+0000","last_clean":"2024-08-20T06:36:02.119747+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:02.119747+0000","last_undegraded":"2024-08-20T06:36:02.119747+0000","last_fullsized":"2024-08-20T06:36:02.119747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125036+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:02.125036+0000","last_peered":"2024-08-20T06:36:02.125036+0000","last_clean":"2024-08-20T06:36:02.125036+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:02.125036+0000","last_undegraded":"2024-08-20T06:36:02.125036+0000","last_fullsized":"2024-08-20T06:36:02.125036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916637+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:35:00.916637+0000","last_peered":"2024-08-20T06:35:00.916637+0000","last_clean":"2024-08-20T06:35:00.916637+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:35:00.916637+0000","last_undegraded":"2024-08-20T06:35:00.916637+0000","last_fullsized":"2024-08-20T06:35:00.916637+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":77,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127854+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:02.127854+0000","last_peered":"2024-08-20T06:36:02.127854+0000","last_clean":"2024-08-20T06:36:02.127854+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:02.127854+0000","last_undegraded":"2024-08-20T06:36:02.127854+0000","last_fullsized":"2024-08-20T06:36:02.127854+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129663+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:03.129663+0000","last_peered":"2024-08-20T06:36:03.129663+0000","last_clean":"2024-08-20T06:36:03.129663+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:03.129663+0000","last_undegraded":"2024-08-20T06:36:03.129663+0000","last_fullsized":"2024-08-20T06:36:03.129663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965368+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:35:07.965368+0000","last_peered":"2024-08-20T06:35:07.965368+0000","last_clean":"2024-08-20T06:35:07.965368+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:35:07.965368+0000","last_undegraded":"2024-08-20T06:35:07.965368+0000","last_fullsized":"2024-08-20T06:35:07.965368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.228019+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:03.228019+0000","last_peered":"2024-08-20T06:36:03.228019+0000","last_clean":"2024-08-20T06:36:03.228019+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:03.228019+0000","last_undegraded":"2024-08-20T06:36:03.228019+0000","last_fullsized":"2024-08-20T06:36:03.228019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129608+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:03.129608+0000","last_peered":"2024-08-20T06:36:03.129608+0000","last_clean":"2024-08-20T06:36:03.129608+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:03.129608+0000","last_undegraded":"2024-08-20T06:36:03.129608+0000","last_fullsized":"2024-08-20T06:36:03.129608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'29","reported_seq":163,"reported_epoch":71,"state":"peering","last_fresh":"2024-08-20T06:36:02.062278+0000","last_change":"2024-08-20T06:36:02.128052+0000","last_active":"2024-08-20T06:36:02.128052+0000","last_peered":"2024-08-20T06:36:02.062278+0000","last_clean":"2024-08-20T06:36:02.062278+0000","last_became_active":"2024-08-20T06:34:57.920116+0000","last_became_peered":"2024-08-20T06:34:57.920116+0000","last_unstale":"2024-08-20T06:36:02.062278+0000","last_undegraded":"2024-08-20T06:36:02.062278+0000","last_fullsized":"2024-08-20T06:36:02.062278+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":112,"num_read_kb":116,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2,4],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.926259+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:35:00.926259+0000","last_peered":"2024-08-20T06:35:00.926259+0000","last_clean":"2024-08-20T06:35:00.926259+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:35:00.926259+0000","last_undegraded":"2024-08-20T06:35:00.926259+0000","last_fullsized":"2024-08-20T06:35:00.926259+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":26,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972040+0000","last_change":"2024-08-20T06:34:57.919137+0000","last_active":"2024-08-20T06:35:08.972040+0000","last_peered":"2024-08-20T06:35:08.972040+0000","last_clean":"2024-08-20T06:35:08.972040+0000","last_became_active":"2024-08-20T06:34:57.918821+0000","last_became_peered":"2024-08-20T06:34:57.918821+0000","last_unstale":"2024-08-20T06:35:08.972040+0000","last_undegraded":"2024-08-20T06:35:08.972040+0000","last_fullsized":"2024-08-20T06:35:08.972040+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:51:18.267609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119821+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:02.119821+0000","last_peered":"2024-08-20T06:36:02.119821+0000","last_clean":"2024-08-20T06:36:02.119821+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:02.119821+0000","last_undegraded":"2024-08-20T06:36:02.119821+0000","last_fullsized":"2024-08-20T06:36:02.119821+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925098+0000","last_change":"2024-08-20T06:34:59.930132+0000","last_active":"2024-08-20T06:35:00.925098+0000","last_peered":"2024-08-20T06:35:00.925098+0000","last_clean":"2024-08-20T06:35:00.925098+0000","last_became_active":"2024-08-20T06:34:59.929820+0000","last_became_peered":"2024-08-20T06:34:59.929820+0000","last_unstale":"2024-08-20T06:35:00.925098+0000","last_undegraded":"2024-08-20T06:35:00.925098+0000","last_fullsized":"2024-08-20T06:35:00.925098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:14:30.947526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":80,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120282+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:02.120282+0000","last_peered":"2024-08-20T06:36:02.120282+0000","last_clean":"2024-08-20T06:36:02.120282+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:02.120282+0000","last_undegraded":"2024-08-20T06:36:02.120282+0000","last_fullsized":"2024-08-20T06:36:02.120282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924673+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:35:00.924673+0000","last_peered":"2024-08-20T06:35:00.924673+0000","last_clean":"2024-08-20T06:35:00.924673+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:35:00.924673+0000","last_undegraded":"2024-08-20T06:35:00.924673+0000","last_fullsized":"2024-08-20T06:35:00.924673+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":35,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973062+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:35:08.973062+0000","last_peered":"2024-08-20T06:35:08.973062+0000","last_clean":"2024-08-20T06:35:08.973062+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:35:08.973062+0000","last_undegraded":"2024-08-20T06:35:08.973062+0000","last_fullsized":"2024-08-20T06:35:08.973062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120911+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:03.120911+0000","last_peered":"2024-08-20T06:36:03.120911+0000","last_clean":"2024-08-20T06:36:03.120911+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:03.120911+0000","last_undegraded":"2024-08-20T06:36:03.120911+0000","last_fullsized":"2024-08-20T06:36:03.120911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:09.980696+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:35:09.980696+0000","last_peered":"2024-08-20T06:35:09.980696+0000","last_clean":"2024-08-20T06:35:09.980696+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:35:09.980696+0000","last_undegraded":"2024-08-20T06:35:09.980696+0000","last_fullsized":"2024-08-20T06:35:09.980696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925884+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:35:00.925884+0000","last_peered":"2024-08-20T06:35:00.925884+0000","last_clean":"2024-08-20T06:35:00.925884+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:35:00.925884+0000","last_undegraded":"2024-08-20T06:35:00.925884+0000","last_fullsized":"2024-08-20T06:35:00.925884+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120080+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:02.120080+0000","last_peered":"2024-08-20T06:36:02.120080+0000","last_clean":"2024-08-20T06:36:02.120080+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:02.120080+0000","last_undegraded":"2024-08-20T06:36:02.120080+0000","last_fullsized":"2024-08-20T06:36:02.120080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925856+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:35:00.925856+0000","last_peered":"2024-08-20T06:35:00.925856+0000","last_clean":"2024-08-20T06:35:00.925856+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:35:00.925856+0000","last_undegraded":"2024-08-20T06:35:00.925856+0000","last_fullsized":"2024-08-20T06:35:00.925856+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":15,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:02.110553+0000","last_change":"2024-08-20T06:34:59.935512+0000","last_active":"2024-08-20T06:35:02.110553+0000","last_peered":"2024-08-20T06:35:02.110553+0000","last_clean":"2024-08-20T06:35:02.110553+0000","last_became_active":"2024-08-20T06:34:59.933714+0000","last_became_peered":"2024-08-20T06:34:59.933714+0000","last_unstale":"2024-08-20T06:35:02.110553+0000","last_undegraded":"2024-08-20T06:35:02.110553+0000","last_fullsized":"2024-08-20T06:35:02.110553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:55:50.042593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.927132+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:35:00.927132+0000","last_peered":"2024-08-20T06:35:00.927132+0000","last_clean":"2024-08-20T06:35:00.927132+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:35:00.927132+0000","last_undegraded":"2024-08-20T06:35:00.927132+0000","last_fullsized":"2024-08-20T06:35:00.927132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924980+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:35:00.924980+0000","last_peered":"2024-08-20T06:35:00.924980+0000","last_clean":"2024-08-20T06:35:00.924980+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:35:00.924980+0000","last_undegraded":"2024-08-20T06:35:00.924980+0000","last_fullsized":"2024-08-20T06:35:00.924980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":26,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120466+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:02.120466+0000","last_peered":"2024-08-20T06:36:02.120466+0000","last_clean":"2024-08-20T06:36:02.120466+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:02.120466+0000","last_undegraded":"2024-08-20T06:36:02.120466+0000","last_fullsized":"2024-08-20T06:36:02.120466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125735+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:02.125735+0000","last_peered":"2024-08-20T06:36:02.125735+0000","last_clean":"2024-08-20T06:36:02.125735+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:02.125735+0000","last_undegraded":"2024-08-20T06:36:02.125735+0000","last_fullsized":"2024-08-20T06:36:02.125735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.972913+0000","last_change":"2024-08-20T06:34:57.914079+0000","last_active":"2024-08-20T06:35:08.972913+0000","last_peered":"2024-08-20T06:35:08.972913+0000","last_clean":"2024-08-20T06:35:08.972913+0000","last_became_active":"2024-08-20T06:34:57.913747+0000","last_became_peered":"2024-08-20T06:34:57.913747+0000","last_unstale":"2024-08-20T06:35:08.972913+0000","last_undegraded":"2024-08-20T06:35:08.972913+0000","last_fullsized":"2024-08-20T06:35:08.972913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:21:24.138487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119732+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:02.119732+0000","last_peered":"2024-08-20T06:36:02.119732+0000","last_clean":"2024-08-20T06:36:02.119732+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:02.119732+0000","last_undegraded":"2024-08-20T06:36:02.119732+0000","last_fullsized":"2024-08-20T06:36:02.119732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":78,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119839+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:02.119839+0000","last_peered":"2024-08-20T06:36:02.119839+0000","last_clean":"2024-08-20T06:36:02.119839+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:02.119839+0000","last_undegraded":"2024-08-20T06:36:02.119839+0000","last_fullsized":"2024-08-20T06:36:02.119839+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120920+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:03.120920+0000","last_peered":"2024-08-20T06:36:03.120920+0000","last_clean":"2024-08-20T06:36:03.120920+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:03.120920+0000","last_undegraded":"2024-08-20T06:36:03.120920+0000","last_fullsized":"2024-08-20T06:36:03.120920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":20,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.925692+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:35:00.925692+0000","last_peered":"2024-08-20T06:35:00.925692+0000","last_clean":"2024-08-20T06:35:00.925692+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:35:00.925692+0000","last_undegraded":"2024-08-20T06:35:00.925692+0000","last_fullsized":"2024-08-20T06:35:00.925692+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127664+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:02.127664+0000","last_peered":"2024-08-20T06:36:02.127664+0000","last_clean":"2024-08-20T06:36:02.127664+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:02.127664+0000","last_undegraded":"2024-08-20T06:36:02.127664+0000","last_fullsized":"2024-08-20T06:36:02.127664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.916869+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:35:00.916869+0000","last_peered":"2024-08-20T06:35:00.916869+0000","last_clean":"2024-08-20T06:35:00.916869+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:35:00.916869+0000","last_undegraded":"2024-08-20T06:35:00.916869+0000","last_fullsized":"2024-08-20T06:35:00.916869+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.127733+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:02.127733+0000","last_peered":"2024-08-20T06:36:02.127733+0000","last_clean":"2024-08-20T06:36:02.127733+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:02.127733+0000","last_undegraded":"2024-08-20T06:36:02.127733+0000","last_fullsized":"2024-08-20T06:36:02.127733+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965120+0000","last_change":"2024-08-20T06:34:57.973505+0000","last_active":"2024-08-20T06:35:07.965120+0000","last_peered":"2024-08-20T06:35:07.965120+0000","last_clean":"2024-08-20T06:35:07.965120+0000","last_became_active":"2024-08-20T06:34:57.973120+0000","last_became_peered":"2024-08-20T06:34:57.973120+0000","last_unstale":"2024-08-20T06:35:07.965120+0000","last_undegraded":"2024-08-20T06:35:07.965120+0000","last_fullsized":"2024-08-20T06:35:07.965120+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:29:02.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120065+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:02.120065+0000","last_peered":"2024-08-20T06:36:02.120065+0000","last_clean":"2024-08-20T06:36:02.120065+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:02.120065+0000","last_undegraded":"2024-08-20T06:36:02.120065+0000","last_fullsized":"2024-08-20T06:36:02.120065+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'10","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:23.924664+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:35:23.924664+0000","last_peered":"2024-08-20T06:35:23.924664+0000","last_clean":"2024-08-20T06:35:23.924664+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:35:23.924664+0000","last_undegraded":"2024-08-20T06:35:23.924664+0000","last_fullsized":"2024-08-20T06:35:23.924664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125987+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:02.125987+0000","last_peered":"2024-08-20T06:36:02.125987+0000","last_clean":"2024-08-20T06:36:02.125987+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:02.125987+0000","last_undegraded":"2024-08-20T06:36:02.125987+0000","last_fullsized":"2024-08-20T06:36:02.125987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120577+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:02.120577+0000","last_peered":"2024-08-20T06:36:02.120577+0000","last_clean":"2024-08-20T06:36:02.120577+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:02.120577+0000","last_undegraded":"2024-08-20T06:36:02.120577+0000","last_fullsized":"2024-08-20T06:36:02.120577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123213+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:03.123213+0000","last_peered":"2024-08-20T06:36:03.123213+0000","last_clean":"2024-08-20T06:36:03.123213+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:03.123213+0000","last_undegraded":"2024-08-20T06:36:03.123213+0000","last_fullsized":"2024-08-20T06:36:03.123213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924239+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:35:00.924239+0000","last_peered":"2024-08-20T06:35:00.924239+0000","last_clean":"2024-08-20T06:35:00.924239+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:35:00.924239+0000","last_undegraded":"2024-08-20T06:35:00.924239+0000","last_fullsized":"2024-08-20T06:35:00.924239+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":24,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125720+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:02.125720+0000","last_peered":"2024-08-20T06:36:02.125720+0000","last_clean":"2024-08-20T06:36:02.125720+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:02.125720+0000","last_undegraded":"2024-08-20T06:36:02.125720+0000","last_fullsized":"2024-08-20T06:36:02.125720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.924929+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:35:00.924929+0000","last_peered":"2024-08-20T06:35:00.924929+0000","last_clean":"2024-08-20T06:35:00.924929+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:35:00.924929+0000","last_undegraded":"2024-08-20T06:35:00.924929+0000","last_fullsized":"2024-08-20T06:35:00.924929+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125796+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:02.125796+0000","last_peered":"2024-08-20T06:36:02.125796+0000","last_clean":"2024-08-20T06:36:02.125796+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:02.125796+0000","last_undegraded":"2024-08-20T06:36:02.125796+0000","last_fullsized":"2024-08-20T06:36:02.125796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":24,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:07.965620+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:35:07.965620+0000","last_peered":"2024-08-20T06:35:07.965620+0000","last_clean":"2024-08-20T06:35:07.965620+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:35:07.965620+0000","last_undegraded":"2024-08-20T06:35:07.965620+0000","last_fullsized":"2024-08-20T06:35:07.965620+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.928848+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:35:00.928848+0000","last_peered":"2024-08-20T06:35:00.928848+0000","last_clean":"2024-08-20T06:35:00.928848+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:35:00.928848+0000","last_undegraded":"2024-08-20T06:35:00.928848+0000","last_fullsized":"2024-08-20T06:35:00.928848+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121464+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:03.121464+0000","last_peered":"2024-08-20T06:36:03.121464+0000","last_clean":"2024-08-20T06:36:03.121464+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:03.121464+0000","last_undegraded":"2024-08-20T06:36:03.121464+0000","last_fullsized":"2024-08-20T06:36:03.121464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":21,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:00.930358+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:35:00.930358+0000","last_peered":"2024-08-20T06:35:00.930358+0000","last_clean":"2024-08-20T06:35:00.930358+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:35:00.930358+0000","last_undegraded":"2024-08-20T06:35:00.930358+0000","last_fullsized":"2024-08-20T06:35:00.930358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129497+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:03.129497+0000","last_peered":"2024-08-20T06:36:03.129497+0000","last_clean":"2024-08-20T06:36:03.129497+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:03.129497+0000","last_undegraded":"2024-08-20T06:36:03.129497+0000","last_fullsized":"2024-08-20T06:36:03.129497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.971790+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:35:08.971790+0000","last_peered":"2024-08-20T06:35:08.971790+0000","last_clean":"2024-08-20T06:35:08.971790+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:35:08.971790+0000","last_undegraded":"2024-08-20T06:35:08.971790+0000","last_fullsized":"2024-08-20T06:35:08.971790+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123408+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:03.123408+0000","last_peered":"2024-08-20T06:36:03.123408+0000","last_clean":"2024-08-20T06:36:03.123408+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:03.123408+0000","last_undegraded":"2024-08-20T06:36:03.123408+0000","last_fullsized":"2024-08-20T06:36:03.123408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.126171+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:02.126171+0000","last_peered":"2024-08-20T06:36:02.126171+0000","last_clean":"2024-08-20T06:36:02.126171+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:02.126171+0000","last_undegraded":"2024-08-20T06:36:02.126171+0000","last_fullsized":"2024-08-20T06:36:02.126171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125991+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:02.125991+0000","last_peered":"2024-08-20T06:36:02.125991+0000","last_clean":"2024-08-20T06:36:02.125991+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:02.125991+0000","last_undegraded":"2024-08-20T06:36:02.125991+0000","last_fullsized":"2024-08-20T06:36:02.125991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-08-20T06:35:08.973139+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:35:08.973139+0000","last_peered":"2024-08-20T06:35:08.973139+0000","last_clean":"2024-08-20T06:35:08.973139+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:35:08.973139+0000","last_undegraded":"2024-08-20T06:35:08.973139+0000","last_fullsized":"2024-08-20T06:35:08.973139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":346,"num_read_kb":240,"num_write":161,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":134296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27772,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717380,"statfs":{"total":95995035648,"available":95966597120,"internally_reserved":0,"allocated":765952,"data_stored":368498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037790,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28452,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716700,"statfs":{"total":95995035648,"available":95965900800,"internally_reserved":0,"allocated":1462272,"data_stored":1021729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168611,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332137,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31948,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713204,"statfs":{"total":95995035648,"available":95962320896,"internally_reserved":0,"allocated":782336,"data_stored":368654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528365,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32528,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712624,"statfs":{"total":95995035648,"available":95961726976,"internally_reserved":0,"allocated":1376256,"data_stored":960193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":376824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1421312,"data_stored":977058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":790528,"data_stored":377261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":368434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607624,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":446847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":806912,"data_stored":378095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:07.979 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:36:08.227 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:08.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 -- 172.21.15.107:0/1369846988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 msgr2=0x7fb55410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:08.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/1369846988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb55410e520 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009ca0 tx=0x7fb54c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 -- 172.21.15.107:0/1369846988 shutdown_connections 2024-08-20T06:36:08.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/1369846988 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb55410e520 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/1369846988 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb5541073a0 0x7fb5541077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/1369846988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb554101990 0x7fb554101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 -- 172.21.15.107:0/1369846988 >> 172.21.15.107:0/1369846988 conn(0x7fb5540fd400 msgr2=0x7fb5540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:08.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 -- 172.21.15.107:0/1369846988 shutdown_connections 2024-08-20T06:36:08.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 -- 172.21.15.107:0/1369846988 wait complete. 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.710+0000 7fb55bf0d640 1 Processor -- start 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 -- start start 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb554101990 0x7fb5540fd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5541073a0 0x7fb5540fde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541118e0 con 0x7fb554107d20 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb554111760 con 0x7fb554101990 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55bf0d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb554111a60 con 0x7fb5541073a0 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54538/0 (socket says 172.21.15.107:54538) 2024-08-20T06:36:08.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 -- 172.21.15.107:0/2463510148 learned_addr learned my addr 172.21.15.107:0/2463510148 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb559ca9640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb554101990 0x7fb5540fd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb5594a8640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5541073a0 0x7fb5540fde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 -- 172.21.15.107:0/2463510148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5541073a0 msgr2=0x7fb5540fde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5541073a0 0x7fb5540fde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 -- 172.21.15.107:0/2463510148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb554101990 msgr2=0x7fb5540fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb554101990 0x7fb5540fd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 -- 172.21.15.107:0/2463510148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5541a7290 con 0x7fb554107d20 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.711+0000 7fb55a4aa640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb54c005cc0 tx=0x7fb54c032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.712+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c032e00 con 0x7fb554107d20 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.712+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb54c0044d0 con 0x7fb554107d20 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.712+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5541a7520 con 0x7fb554107d20 2024-08-20T06:36:08.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.712+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c030b70 con 0x7fb554107d20 2024-08-20T06:36:08.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.712+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5541a7ab0 con 0x7fb554107d20 2024-08-20T06:36:08.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.713+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb554101d70 con 0x7fb554107d20 2024-08-20T06:36:08.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.714+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fb54c030d10 con 0x7fb554107d20 2024-08-20T06:36:08.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.715+0000 7fb546ffd640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 0x7fb5400b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:08.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.715+0000 7fb559ca9640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 0x7fb5400b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.715+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb54c105190 con 0x7fb554107d20 2024-08-20T06:36:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.715+0000 7fb559ca9640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 0x7fb5400b61a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb55000bbe0 tx=0x7fb55000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:08.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.718+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb54c0cc820 con 0x7fb554107d20 2024-08-20T06:36:08.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.890+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb55410ab70 con 0x7fb5400b3ce0 2024-08-20T06:36:08.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.894+0000 7fb546ffd640 1 -- 172.21.15.107:0/2463510148 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7fb55410ab70 con 0x7fb5400b3ce0 2024-08-20T06:36:08.898 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:08.899 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:08.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 msgr2=0x7fb5400b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:08.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 0x7fb5400b61a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb55000bbe0 tx=0x7fb55000b390 comp rx=0 tx=0).stop 2024-08-20T06:36:08.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 msgr2=0x7fb5540fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb54c005cc0 tx=0x7fb54c032a10 comp rx=0 tx=0).stop 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 shutdown_connections 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb554107d20 0x7fb5540fe380 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb5400b3ce0 0x7fb5400b61a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5541073a0 0x7fb5540fde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 --2- 172.21.15.107:0/2463510148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb554101990 0x7fb5540fd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 >> 172.21.15.107:0/2463510148 conn(0x7fb5540fd400 msgr2=0x7fb55406aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 shutdown_connections 2024-08-20T06:36:08.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:08.897+0000 7fb55bf0d640 1 -- 172.21.15.107:0/2463510148 wait complete. 2024-08-20T06:36:09.204 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:09 smithi107 ceph-mon[18495]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:09.204 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:09 smithi107 ceph-mon[18495]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:09.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:09 smithi183 ceph-mon[22204]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:09.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:09 smithi183 ceph-mon[22204]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:09.516 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-08-20T06:36:07.321523+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":434,"num_read_kb":312,"num_write":286,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368964,"kb_used_data":11340,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572860,"statfs":{"total":1151940427776,"available":1151562608640,"internally_reserved":0,"allocated":11612160,"data_stored":6484796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":22,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":61923,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001727"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125647+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:03.125647+0000","last_peered":"2024-08-20T06:36:03.125647+0000","last_clean":"2024-08-20T06:36:03.125647+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:03.125647+0000","last_undegraded":"2024-08-20T06:36:03.125647+0000","last_fullsized":"2024-08-20T06:36:03.125647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123128+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:03.123128+0000","last_peered":"2024-08-20T06:36:03.123128+0000","last_clean":"2024-08-20T06:36:03.123128+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:03.123128+0000","last_undegraded":"2024-08-20T06:36:03.123128+0000","last_fullsized":"2024-08-20T06:36:03.123128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123174+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:03.123174+0000","last_peered":"2024-08-20T06:36:03.123174+0000","last_clean":"2024-08-20T06:36:03.123174+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:03.123174+0000","last_undegraded":"2024-08-20T06:36:03.123174+0000","last_fullsized":"2024-08-20T06:36:03.123174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.127705+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:03.127705+0000","last_peered":"2024-08-20T06:36:03.127705+0000","last_clean":"2024-08-20T06:36:03.127705+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:03.127705+0000","last_undegraded":"2024-08-20T06:36:03.127705+0000","last_fullsized":"2024-08-20T06:36:03.127705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125056+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:03.125056+0000","last_peered":"2024-08-20T06:36:03.125056+0000","last_clean":"2024-08-20T06:36:03.125056+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:03.125056+0000","last_undegraded":"2024-08-20T06:36:03.125056+0000","last_fullsized":"2024-08-20T06:36:03.125056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124679+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:03.124679+0000","last_peered":"2024-08-20T06:36:03.124679+0000","last_clean":"2024-08-20T06:36:03.124679+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:03.124679+0000","last_undegraded":"2024-08-20T06:36:03.124679+0000","last_fullsized":"2024-08-20T06:36:03.124679+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126920+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:03.126920+0000","last_peered":"2024-08-20T06:36:03.126920+0000","last_clean":"2024-08-20T06:36:03.126920+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:03.126920+0000","last_undegraded":"2024-08-20T06:36:03.126920+0000","last_fullsized":"2024-08-20T06:36:03.126920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.124966+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:02.124966+0000","last_peered":"2024-08-20T06:36:02.124966+0000","last_clean":"2024-08-20T06:36:02.124966+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:02.124966+0000","last_undegraded":"2024-08-20T06:36:02.124966+0000","last_fullsized":"2024-08-20T06:36:02.124966+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129780+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:03.129780+0000","last_peered":"2024-08-20T06:36:03.129780+0000","last_clean":"2024-08-20T06:36:03.129780+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:03.129780+0000","last_undegraded":"2024-08-20T06:36:03.129780+0000","last_fullsized":"2024-08-20T06:36:03.129780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125319+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:03.125319+0000","last_peered":"2024-08-20T06:36:03.125319+0000","last_clean":"2024-08-20T06:36:03.125319+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:03.125319+0000","last_undegraded":"2024-08-20T06:36:03.125319+0000","last_fullsized":"2024-08-20T06:36:03.125319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122876+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:03.122876+0000","last_peered":"2024-08-20T06:36:03.122876+0000","last_clean":"2024-08-20T06:36:03.122876+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:03.122876+0000","last_undegraded":"2024-08-20T06:36:03.122876+0000","last_fullsized":"2024-08-20T06:36:03.122876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120512+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:02.120512+0000","last_peered":"2024-08-20T06:36:02.120512+0000","last_clean":"2024-08-20T06:36:02.120512+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:02.120512+0000","last_undegraded":"2024-08-20T06:36:02.120512+0000","last_fullsized":"2024-08-20T06:36:02.120512+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120134+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:02.120134+0000","last_peered":"2024-08-20T06:36:02.120134+0000","last_clean":"2024-08-20T06:36:02.120134+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:02.120134+0000","last_undegraded":"2024-08-20T06:36:02.120134+0000","last_fullsized":"2024-08-20T06:36:02.120134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122863+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:03.122863+0000","last_peered":"2024-08-20T06:36:03.122863+0000","last_clean":"2024-08-20T06:36:03.122863+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:03.122863+0000","last_undegraded":"2024-08-20T06:36:03.122863+0000","last_fullsized":"2024-08-20T06:36:03.122863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122802+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:03.122802+0000","last_peered":"2024-08-20T06:36:03.122802+0000","last_clean":"2024-08-20T06:36:03.122802+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:03.122802+0000","last_undegraded":"2024-08-20T06:36:03.122802+0000","last_fullsized":"2024-08-20T06:36:03.122802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131241+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:03.131241+0000","last_peered":"2024-08-20T06:36:03.131241+0000","last_clean":"2024-08-20T06:36:03.131241+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:03.131241+0000","last_undegraded":"2024-08-20T06:36:03.131241+0000","last_fullsized":"2024-08-20T06:36:03.131241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125701+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:03.125701+0000","last_peered":"2024-08-20T06:36:03.125701+0000","last_clean":"2024-08-20T06:36:03.125701+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:03.125701+0000","last_undegraded":"2024-08-20T06:36:03.125701+0000","last_fullsized":"2024-08-20T06:36:03.125701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123606+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:03.123606+0000","last_peered":"2024-08-20T06:36:03.123606+0000","last_clean":"2024-08-20T06:36:03.123606+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:03.123606+0000","last_undegraded":"2024-08-20T06:36:03.123606+0000","last_fullsized":"2024-08-20T06:36:03.123606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122107+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:03.122107+0000","last_peered":"2024-08-20T06:36:03.122107+0000","last_clean":"2024-08-20T06:36:03.122107+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:03.122107+0000","last_undegraded":"2024-08-20T06:36:03.122107+0000","last_fullsized":"2024-08-20T06:36:03.122107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125244+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:03.125244+0000","last_peered":"2024-08-20T06:36:03.125244+0000","last_clean":"2024-08-20T06:36:03.125244+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:03.125244+0000","last_undegraded":"2024-08-20T06:36:03.125244+0000","last_fullsized":"2024-08-20T06:36:03.125244+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120192+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:02.120192+0000","last_peered":"2024-08-20T06:36:02.120192+0000","last_clean":"2024-08-20T06:36:02.120192+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:02.120192+0000","last_undegraded":"2024-08-20T06:36:02.120192+0000","last_fullsized":"2024-08-20T06:36:02.120192+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129402+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:03.129402+0000","last_peered":"2024-08-20T06:36:03.129402+0000","last_clean":"2024-08-20T06:36:03.129402+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:03.129402+0000","last_undegraded":"2024-08-20T06:36:03.129402+0000","last_fullsized":"2024-08-20T06:36:03.129402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125614+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:02.125614+0000","last_peered":"2024-08-20T06:36:02.125614+0000","last_clean":"2024-08-20T06:36:02.125614+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:02.125614+0000","last_undegraded":"2024-08-20T06:36:02.125614+0000","last_fullsized":"2024-08-20T06:36:02.125614+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120983+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:03.120983+0000","last_peered":"2024-08-20T06:36:03.120983+0000","last_clean":"2024-08-20T06:36:03.120983+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:03.120983+0000","last_undegraded":"2024-08-20T06:36:03.120983+0000","last_fullsized":"2024-08-20T06:36:03.120983+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125871+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:02.125871+0000","last_peered":"2024-08-20T06:36:02.125871+0000","last_clean":"2024-08-20T06:36:02.125871+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:02.125871+0000","last_undegraded":"2024-08-20T06:36:02.125871+0000","last_fullsized":"2024-08-20T06:36:02.125871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122256+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:03.122256+0000","last_peered":"2024-08-20T06:36:03.122256+0000","last_clean":"2024-08-20T06:36:03.122256+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:03.122256+0000","last_undegraded":"2024-08-20T06:36:03.122256+0000","last_fullsized":"2024-08-20T06:36:03.122256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126240+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:03.126240+0000","last_peered":"2024-08-20T06:36:03.126240+0000","last_clean":"2024-08-20T06:36:03.126240+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:03.126240+0000","last_undegraded":"2024-08-20T06:36:03.126240+0000","last_fullsized":"2024-08-20T06:36:03.126240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120039+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:02.120039+0000","last_peered":"2024-08-20T06:36:02.120039+0000","last_clean":"2024-08-20T06:36:02.120039+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:02.120039+0000","last_undegraded":"2024-08-20T06:36:02.120039+0000","last_fullsized":"2024-08-20T06:36:02.120039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123039+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:03.123039+0000","last_peered":"2024-08-20T06:36:03.123039+0000","last_clean":"2024-08-20T06:36:03.123039+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:03.123039+0000","last_undegraded":"2024-08-20T06:36:03.123039+0000","last_fullsized":"2024-08-20T06:36:03.123039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121952+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:03.121952+0000","last_peered":"2024-08-20T06:36:03.121952+0000","last_clean":"2024-08-20T06:36:03.121952+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:03.121952+0000","last_undegraded":"2024-08-20T06:36:03.121952+0000","last_fullsized":"2024-08-20T06:36:03.121952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122088+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:03.122088+0000","last_peered":"2024-08-20T06:36:03.122088+0000","last_clean":"2024-08-20T06:36:03.122088+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:03.122088+0000","last_undegraded":"2024-08-20T06:36:03.122088+0000","last_fullsized":"2024-08-20T06:36:03.122088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125069+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:03.125069+0000","last_peered":"2024-08-20T06:36:03.125069+0000","last_clean":"2024-08-20T06:36:03.125069+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:03.125069+0000","last_undegraded":"2024-08-20T06:36:03.125069+0000","last_fullsized":"2024-08-20T06:36:03.125069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124791+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:03.124791+0000","last_peered":"2024-08-20T06:36:03.124791+0000","last_clean":"2024-08-20T06:36:03.124791+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:03.124791+0000","last_undegraded":"2024-08-20T06:36:03.124791+0000","last_fullsized":"2024-08-20T06:36:03.124791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":86,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:04.065723+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:04.065723+0000","last_peered":"2024-08-20T06:36:04.065723+0000","last_clean":"2024-08-20T06:36:04.065723+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:04.065723+0000","last_undegraded":"2024-08-20T06:36:04.065723+0000","last_fullsized":"2024-08-20T06:36:04.065723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121272+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:03.121272+0000","last_peered":"2024-08-20T06:36:03.121272+0000","last_clean":"2024-08-20T06:36:03.121272+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:03.121272+0000","last_undegraded":"2024-08-20T06:36:03.121272+0000","last_fullsized":"2024-08-20T06:36:03.121272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119747+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:02.119747+0000","last_peered":"2024-08-20T06:36:02.119747+0000","last_clean":"2024-08-20T06:36:02.119747+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:02.119747+0000","last_undegraded":"2024-08-20T06:36:02.119747+0000","last_fullsized":"2024-08-20T06:36:02.119747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125036+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:02.125036+0000","last_peered":"2024-08-20T06:36:02.125036+0000","last_clean":"2024-08-20T06:36:02.125036+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:02.125036+0000","last_undegraded":"2024-08-20T06:36:02.125036+0000","last_fullsized":"2024-08-20T06:36:02.125036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123444+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:03.123444+0000","last_peered":"2024-08-20T06:36:03.123444+0000","last_clean":"2024-08-20T06:36:03.123444+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:03.123444+0000","last_undegraded":"2024-08-20T06:36:03.123444+0000","last_fullsized":"2024-08-20T06:36:03.123444+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":84,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:06.067915+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:06.067915+0000","last_peered":"2024-08-20T06:36:06.067915+0000","last_clean":"2024-08-20T06:36:06.067915+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:06.067915+0000","last_undegraded":"2024-08-20T06:36:06.067915+0000","last_fullsized":"2024-08-20T06:36:06.067915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129663+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:03.129663+0000","last_peered":"2024-08-20T06:36:03.129663+0000","last_clean":"2024-08-20T06:36:03.129663+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:03.129663+0000","last_undegraded":"2024-08-20T06:36:03.129663+0000","last_fullsized":"2024-08-20T06:36:03.129663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.228019+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:03.228019+0000","last_peered":"2024-08-20T06:36:03.228019+0000","last_clean":"2024-08-20T06:36:03.228019+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:03.228019+0000","last_undegraded":"2024-08-20T06:36:03.228019+0000","last_fullsized":"2024-08-20T06:36:03.228019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129608+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:03.129608+0000","last_peered":"2024-08-20T06:36:03.129608+0000","last_clean":"2024-08-20T06:36:03.129608+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:03.129608+0000","last_undegraded":"2024-08-20T06:36:03.129608+0000","last_fullsized":"2024-08-20T06:36:03.129608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'29","reported_seq":193,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:06.348346+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:06.348346+0000","last_peered":"2024-08-20T06:36:06.348346+0000","last_clean":"2024-08-20T06:36:06.348346+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:06.348346+0000","last_undegraded":"2024-08-20T06:36:06.348346+0000","last_fullsized":"2024-08-20T06:36:06.348346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126395+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:03.126395+0000","last_peered":"2024-08-20T06:36:03.126395+0000","last_clean":"2024-08-20T06:36:03.126395+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:03.126395+0000","last_undegraded":"2024-08-20T06:36:03.126395+0000","last_fullsized":"2024-08-20T06:36:03.126395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119821+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:02.119821+0000","last_peered":"2024-08-20T06:36:02.119821+0000","last_clean":"2024-08-20T06:36:02.119821+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:02.119821+0000","last_undegraded":"2024-08-20T06:36:02.119821+0000","last_fullsized":"2024-08-20T06:36:02.119821+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227994+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:03.227994+0000","last_peered":"2024-08-20T06:36:03.227994+0000","last_clean":"2024-08-20T06:36:03.227994+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:03.227994+0000","last_undegraded":"2024-08-20T06:36:03.227994+0000","last_fullsized":"2024-08-20T06:36:03.227994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":80,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120282+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:02.120282+0000","last_peered":"2024-08-20T06:36:02.120282+0000","last_clean":"2024-08-20T06:36:02.120282+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:02.120282+0000","last_undegraded":"2024-08-20T06:36:02.120282+0000","last_fullsized":"2024-08-20T06:36:02.120282+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":57,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121969+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:03.121969+0000","last_peered":"2024-08-20T06:36:03.121969+0000","last_clean":"2024-08-20T06:36:03.121969+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:03.121969+0000","last_undegraded":"2024-08-20T06:36:03.121969+0000","last_fullsized":"2024-08-20T06:36:03.121969+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120911+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:03.120911+0000","last_peered":"2024-08-20T06:36:03.120911+0000","last_clean":"2024-08-20T06:36:03.120911+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:03.120911+0000","last_undegraded":"2024-08-20T06:36:03.120911+0000","last_fullsized":"2024-08-20T06:36:03.120911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125364+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:03.125364+0000","last_peered":"2024-08-20T06:36:03.125364+0000","last_clean":"2024-08-20T06:36:03.125364+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:03.125364+0000","last_undegraded":"2024-08-20T06:36:03.125364+0000","last_fullsized":"2024-08-20T06:36:03.125364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120080+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:02.120080+0000","last_peered":"2024-08-20T06:36:02.120080+0000","last_clean":"2024-08-20T06:36:02.120080+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:02.120080+0000","last_undegraded":"2024-08-20T06:36:02.120080+0000","last_fullsized":"2024-08-20T06:36:02.120080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122921+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:03.122921+0000","last_peered":"2024-08-20T06:36:03.122921+0000","last_clean":"2024-08-20T06:36:03.122921+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:03.122921+0000","last_undegraded":"2024-08-20T06:36:03.122921+0000","last_fullsized":"2024-08-20T06:36:03.122921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.132123+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:03.132123+0000","last_peered":"2024-08-20T06:36:03.132123+0000","last_clean":"2024-08-20T06:36:03.132123+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:03.132123+0000","last_undegraded":"2024-08-20T06:36:03.132123+0000","last_fullsized":"2024-08-20T06:36:03.132123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123036+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:03.123036+0000","last_peered":"2024-08-20T06:36:03.123036+0000","last_clean":"2024-08-20T06:36:03.123036+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:03.123036+0000","last_undegraded":"2024-08-20T06:36:03.123036+0000","last_fullsized":"2024-08-20T06:36:03.123036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122598+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122598+0000","last_peered":"2024-08-20T06:36:03.122598+0000","last_clean":"2024-08-20T06:36:03.122598+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122598+0000","last_undegraded":"2024-08-20T06:36:03.122598+0000","last_fullsized":"2024-08-20T06:36:03.122598+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":26,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120466+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:02.120466+0000","last_peered":"2024-08-20T06:36:02.120466+0000","last_clean":"2024-08-20T06:36:02.120466+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:02.120466+0000","last_undegraded":"2024-08-20T06:36:02.120466+0000","last_fullsized":"2024-08-20T06:36:02.120466+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125735+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:02.125735+0000","last_peered":"2024-08-20T06:36:02.125735+0000","last_clean":"2024-08-20T06:36:02.125735+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:02.125735+0000","last_undegraded":"2024-08-20T06:36:02.125735+0000","last_fullsized":"2024-08-20T06:36:02.125735+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.134693+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.134693+0000","last_peered":"2024-08-20T06:36:03.134693+0000","last_clean":"2024-08-20T06:36:03.134693+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.134693+0000","last_undegraded":"2024-08-20T06:36:03.134693+0000","last_fullsized":"2024-08-20T06:36:03.134693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119732+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:02.119732+0000","last_peered":"2024-08-20T06:36:02.119732+0000","last_clean":"2024-08-20T06:36:02.119732+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:02.119732+0000","last_undegraded":"2024-08-20T06:36:02.119732+0000","last_fullsized":"2024-08-20T06:36:02.119732+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":78,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.119839+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:02.119839+0000","last_peered":"2024-08-20T06:36:02.119839+0000","last_clean":"2024-08-20T06:36:02.119839+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:02.119839+0000","last_undegraded":"2024-08-20T06:36:02.119839+0000","last_fullsized":"2024-08-20T06:36:02.119839+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120920+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:03.120920+0000","last_peered":"2024-08-20T06:36:03.120920+0000","last_clean":"2024-08-20T06:36:03.120920+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:03.120920+0000","last_undegraded":"2024-08-20T06:36:03.120920+0000","last_fullsized":"2024-08-20T06:36:03.120920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125208+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:03.125208+0000","last_peered":"2024-08-20T06:36:03.125208+0000","last_clean":"2024-08-20T06:36:03.125208+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:03.125208+0000","last_undegraded":"2024-08-20T06:36:03.125208+0000","last_fullsized":"2024-08-20T06:36:03.125208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124981+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:03.124981+0000","last_peered":"2024-08-20T06:36:03.124981+0000","last_clean":"2024-08-20T06:36:03.124981+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:03.124981+0000","last_undegraded":"2024-08-20T06:36:03.124981+0000","last_fullsized":"2024-08-20T06:36:03.124981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120065+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:02.120065+0000","last_peered":"2024-08-20T06:36:02.120065+0000","last_clean":"2024-08-20T06:36:02.120065+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:02.120065+0000","last_undegraded":"2024-08-20T06:36:02.120065+0000","last_fullsized":"2024-08-20T06:36:02.120065+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'10","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123630+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123630+0000","last_peered":"2024-08-20T06:36:03.123630+0000","last_clean":"2024-08-20T06:36:03.123630+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123630+0000","last_undegraded":"2024-08-20T06:36:03.123630+0000","last_fullsized":"2024-08-20T06:36:03.123630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125987+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:02.125987+0000","last_peered":"2024-08-20T06:36:02.125987+0000","last_clean":"2024-08-20T06:36:02.125987+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:02.125987+0000","last_undegraded":"2024-08-20T06:36:02.125987+0000","last_fullsized":"2024-08-20T06:36:02.125987+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":25,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.120577+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:02.120577+0000","last_peered":"2024-08-20T06:36:02.120577+0000","last_clean":"2024-08-20T06:36:02.120577+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:02.120577+0000","last_undegraded":"2024-08-20T06:36:02.120577+0000","last_fullsized":"2024-08-20T06:36:02.120577+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123213+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:03.123213+0000","last_peered":"2024-08-20T06:36:03.123213+0000","last_clean":"2024-08-20T06:36:03.123213+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:03.123213+0000","last_undegraded":"2024-08-20T06:36:03.123213+0000","last_fullsized":"2024-08-20T06:36:03.123213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124745+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:03.124745+0000","last_peered":"2024-08-20T06:36:03.124745+0000","last_clean":"2024-08-20T06:36:03.124745+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:03.124745+0000","last_undegraded":"2024-08-20T06:36:03.124745+0000","last_fullsized":"2024-08-20T06:36:03.124745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":24,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125720+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:02.125720+0000","last_peered":"2024-08-20T06:36:02.125720+0000","last_clean":"2024-08-20T06:36:02.125720+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:02.125720+0000","last_undegraded":"2024-08-20T06:36:02.125720+0000","last_fullsized":"2024-08-20T06:36:02.125720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125248+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:03.125248+0000","last_peered":"2024-08-20T06:36:03.125248+0000","last_clean":"2024-08-20T06:36:03.125248+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:03.125248+0000","last_undegraded":"2024-08-20T06:36:03.125248+0000","last_fullsized":"2024-08-20T06:36:03.125248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125796+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:02.125796+0000","last_peered":"2024-08-20T06:36:02.125796+0000","last_clean":"2024-08-20T06:36:02.125796+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:02.125796+0000","last_undegraded":"2024-08-20T06:36:02.125796+0000","last_fullsized":"2024-08-20T06:36:02.125796+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122092+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:03.122092+0000","last_peered":"2024-08-20T06:36:03.122092+0000","last_clean":"2024-08-20T06:36:03.122092+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:03.122092+0000","last_undegraded":"2024-08-20T06:36:03.122092+0000","last_fullsized":"2024-08-20T06:36:03.122092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121464+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:03.121464+0000","last_peered":"2024-08-20T06:36:03.121464+0000","last_clean":"2024-08-20T06:36:03.121464+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:03.121464+0000","last_undegraded":"2024-08-20T06:36:03.121464+0000","last_fullsized":"2024-08-20T06:36:03.121464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129497+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:03.129497+0000","last_peered":"2024-08-20T06:36:03.129497+0000","last_clean":"2024-08-20T06:36:03.129497+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:03.129497+0000","last_undegraded":"2024-08-20T06:36:03.129497+0000","last_fullsized":"2024-08-20T06:36:03.129497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123408+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:03.123408+0000","last_peered":"2024-08-20T06:36:03.123408+0000","last_clean":"2024-08-20T06:36:03.123408+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:03.123408+0000","last_undegraded":"2024-08-20T06:36:03.123408+0000","last_fullsized":"2024-08-20T06:36:03.123408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":23,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.126171+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:02.126171+0000","last_peered":"2024-08-20T06:36:02.126171+0000","last_clean":"2024-08-20T06:36:02.126171+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:02.126171+0000","last_undegraded":"2024-08-20T06:36:02.126171+0000","last_fullsized":"2024-08-20T06:36:02.126171+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-08-20T06:36:02.125991+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:02.125991+0000","last_peered":"2024-08-20T06:36:02.125991+0000","last_clean":"2024-08-20T06:36:02.125991+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:02.125991+0000","last_undegraded":"2024-08-20T06:36:02.125991+0000","last_fullsized":"2024-08-20T06:36:02.125991+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":358,"num_read_kb":248,"num_write":161,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":134330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037791,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1495040,"data_stored":1038910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168612,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332138,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528366,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1409024,"data_stored":977374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724593,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27796,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717356,"statfs":{"total":95995035648,"available":95966572544,"internally_reserved":0,"allocated":786432,"data_stored":376824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888119,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1421312,"data_stored":977058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":790528,"data_stored":377261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247871,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27792,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717360,"statfs":{"total":95995035648,"available":95966576640,"internally_reserved":0,"allocated":786432,"data_stored":368434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444102,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607624,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":446847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738446,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":806912,"data_stored":378095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:09.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph status --format=json 2024-08-20T06:36:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:09 smithi193 ceph-mon[21944]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:09 smithi193 ceph-mon[21944]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:09.774 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.286+0000 7f71bedfa640 1 -- 172.21.15.107:0/3509010207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 msgr2=0x7f71b81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.286+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3509010207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b81077e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009900 tx=0x7f71a402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.286+0000 7f71bedfa640 1 -- 172.21.15.107:0/3509010207 shutdown_connections 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.286+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3509010207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 0x7f71b810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.286+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3509010207 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b81077e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3509010207 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 -- 172.21.15.107:0/3509010207 >> 172.21.15.107:0/3509010207 conn(0x7f71b80fd400 msgr2=0x7f71b80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:10.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 -- 172.21.15.107:0/3509010207 shutdown_connections 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 -- 172.21.15.107:0/3509010207 wait complete. 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 Processor -- start 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.287+0000 7f71bedfa640 1 -- start start 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 0x7f71b819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f71b81118d0 con 0x7f71b81073a0 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8111750 con 0x7f71b8101990 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f71b8111a50 con 0x7f71b8107d20 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54550/0 (socket says 172.21.15.107:54550) 2024-08-20T06:36:10.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 -- 172.21.15.107:0/3724244957 learned_addr learned my addr 172.21.15.107:0/3724244957 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bd397640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 0x7f71b819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bcb96640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 -- 172.21.15.107:0/3724244957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 msgr2=0x7f71b819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 0x7f71b819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 -- 172.21.15.107:0/3724244957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 msgr2=0x7f71b8199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 -- 172.21.15.107:0/3724244957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b81a5220 con 0x7f71b81073a0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bcb96640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b7fff640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009a20 tx=0x7f71a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a403f030 con 0x7f71b81073a0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b81a54b0 con 0x7f71b81073a0 2024-08-20T06:36:10.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.288+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f71a40035f0 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.289+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4035070 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.289+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b81a5a10 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.290+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b8101df0 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.293+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f71a4003aa0 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.293+0000 7f71b5ffb640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 0x7f71a80b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.294+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f71a4104c80 con 0x7f71b81073a0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.294+0000 7f71bcb96640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 0x7f71a80b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.294+0000 7f71bcb96640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 0x7f71a80b6120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009930 tx=0x7f71ac008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:10.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.295+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f71a40cc170 con 0x7f71b81073a0 2024-08-20T06:36:10.451 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:10 smithi183 ceph-mon[22204]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:10.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.545+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f71b806abc0 con 0x7f71b81073a0 2024-08-20T06:36:10.547 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:10 smithi107 ceph-mon[18495]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:10.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.546+0000 7f71b5ffb640 1 -- 172.21.15.107:0/3724244957 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2361 (secure 0 0 0) 0x7f71a40cba70 con 0x7f71b81073a0 2024-08-20T06:36:10.547 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:10.547 INFO:teuthology.orchestra.run.smithi107.stdout:{"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":441,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":73,"num_osds":12,"num_up_osds":12,"osd_up_since":1724135650,"num_in_osds":12,"osd_in_since":1724135628,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":64,"data_bytes":655962,"bytes_used":377982976,"bytes_avail":1151562444800,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":7738,"recovering_keys_per_sec":1,"num_objects_recovered":12,"num_bytes_recovered":61923,"num_keys_recovered":10,"read_bytes_sec":3455,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24533},{"filesystem_id":1,"rank":1,"name":"f","status":"up:standby-replay","gid":24541},{"filesystem_id":1,"rank":0,"name":"c","status":"up:standby-replay","gid":24547},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24548},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24562},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24572}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.107:8443/"}},"servicemap":{"epoch":25,"modified":"2024-08-20T06:36:07.321742+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-08-20T06:36:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 msgr2=0x7f71a80b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 0x7f71a80b6120 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f71ac009930 tx=0x7f71ac008040 comp rx=0 tx=0).stop 2024-08-20T06:36:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 msgr2=0x7f71b819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:10.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009a20 tx=0x7f71a4004270 comp rx=0 tx=0).stop 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 shutdown_connections 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f71a80b3c60 0x7f71a80b6120 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b8107d20 0x7f71b819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71b81073a0 0x7f71b819a380 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 --2- 172.21.15.107:0/3724244957 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f71b8101990 0x7f71b8199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 >> 172.21.15.107:0/3724244957 conn(0x7f71b80fd400 msgr2=0x7f71b80ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 shutdown_connections 2024-08-20T06:36:10.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:10.548+0000 7f71bedfa640 1 -- 172.21.15.107:0/3724244957 wait complete. 2024-08-20T06:36:10.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:10 smithi193 ceph-mon[21944]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:11.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:11 smithi107 ceph-mon[18495]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:11.229 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:11 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3724244957' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T06:36:11.256 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-08-20T06:36:11.256 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:36:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:11 smithi183 ceph-mon[22204]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:11 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3724244957' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T06:36:11.504 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:11.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:11 smithi193 ceph-mon[21944]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 7.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:11.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:11 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3724244957' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T06:36:11.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- 172.21.15.107:0/1971590936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76001073a0 msgr2=0x7f76001077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 --2- 172.21.15.107:0/1971590936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76001073a0 0x7f76001077e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009900 tx=0x7f75ec02a410 comp rx=0 tx=0).stop 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- 172.21.15.107:0/1971590936 shutdown_connections 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 --2- 172.21.15.107:0/1971590936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7600107d20 0x7f760010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 --2- 172.21.15.107:0/1971590936 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f76001073a0 0x7f76001077e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 --2- 172.21.15.107:0/1971590936 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7600101990 0x7f7600101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- 172.21.15.107:0/1971590936 >> 172.21.15.107:0/1971590936 conn(0x7f76000fd400 msgr2=0x7f76000ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- 172.21.15.107:0/1971590936 shutdown_connections 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- 172.21.15.107:0/1971590936 wait complete. 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 Processor -- start 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.981+0000 7f760706e640 1 -- start start 2024-08-20T06:36:11.984 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76001073a0 0x7f7600198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7600107d20 0x7f7600198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7600111850 con 0x7f7600101990 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f76001116d0 con 0x7f7600107d20 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f76001119d0 con 0x7f76001073a0 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:54564/0 (socket says 172.21.15.107:54564) 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 -- 172.21.15.107:0/1979714703 learned_addr learned my addr 172.21.15.107:0/1979714703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 -- 172.21.15.107:0/1979714703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76001073a0 msgr2=0x7f7600198010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76001073a0 0x7f7600198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:11.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 -- 172.21.15.107:0/1979714703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7600107d20 msgr2=0x7f7600198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7600107d20 0x7f7600198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 -- 172.21.15.107:0/1979714703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76001a72b0 con 0x7f7600101990 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f7604e0a640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75f400d950 tx=0x7f75f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4017070 con 0x7f7600101990 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76001a75a0 con 0x7f7600101990 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f75f4009c30 con 0x7f7600101990 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.982+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f400f620 con 0x7f7600101990 2024-08-20T06:36:11.986 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.983+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76001a7b30 con 0x7f7600101990 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.984+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7600101d70 con 0x7f7600101990 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.985+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f75f4010460 con 0x7f7600101990 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.985+0000 7f75fdffb640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 0x7f75f00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.986+0000 7f75fffff640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 0x7f75f00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.986+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f75f40e3a70 con 0x7f7600101990 2024-08-20T06:36:11.987 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.986+0000 7f75fffff640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 0x7f75f00b6120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75ec005c20 tx=0x7f75ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:11.990 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:11.989+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f75f40aaf60 con 0x7f7600101990 2024-08-20T06:36:12.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.167+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f760002e3d0 con 0x7f75f00b3c60 2024-08-20T06:36:12.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.171+0000 7f75fdffb640 1 -- 172.21.15.107:0/1979714703 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258729 (secure 0 0 0) 0x7f760002e3d0 con 0x7f75f00b3c60 2024-08-20T06:36:12.174 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:12.175 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:12.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 msgr2=0x7f75f00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:12.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 0x7f75f00b6120 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75ec005c20 tx=0x7f75ec000bc0 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 msgr2=0x7f7600197ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75f400d950 tx=0x7f75f400de10 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 shutdown_connections 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7600107d20 0x7f7600198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f75f00b3c60 0x7f75f00b6120 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76001073a0 0x7f7600198010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 --2- 172.21.15.107:0/1979714703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7600101990 0x7f7600197ad0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 >> 172.21.15.107:0/1979714703 conn(0x7f76000fd400 msgr2=0x7f76000ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 shutdown_connections 2024-08-20T06:36:12.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:12.175+0000 7f760706e640 1 -- 172.21.15.107:0/1979714703 wait complete. 2024-08-20T06:36:12.914 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-08-20T06:36:11.322705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":449,"num_read_kb":317,"num_write":286,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369124,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572700,"statfs":{"total":1151940427776,"available":1151562444800,"internally_reserved":0,"allocated":11759616,"data_stored":6589050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":55,"num_read_kb":27,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":61923,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.204236"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125647+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:03.125647+0000","last_peered":"2024-08-20T06:36:03.125647+0000","last_clean":"2024-08-20T06:36:03.125647+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:03.125647+0000","last_undegraded":"2024-08-20T06:36:03.125647+0000","last_fullsized":"2024-08-20T06:36:03.125647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123128+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:03.123128+0000","last_peered":"2024-08-20T06:36:03.123128+0000","last_clean":"2024-08-20T06:36:03.123128+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:03.123128+0000","last_undegraded":"2024-08-20T06:36:03.123128+0000","last_fullsized":"2024-08-20T06:36:03.123128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123174+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:03.123174+0000","last_peered":"2024-08-20T06:36:03.123174+0000","last_clean":"2024-08-20T06:36:03.123174+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:03.123174+0000","last_undegraded":"2024-08-20T06:36:03.123174+0000","last_fullsized":"2024-08-20T06:36:03.123174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.127705+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:03.127705+0000","last_peered":"2024-08-20T06:36:03.127705+0000","last_clean":"2024-08-20T06:36:03.127705+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:03.127705+0000","last_undegraded":"2024-08-20T06:36:03.127705+0000","last_fullsized":"2024-08-20T06:36:03.127705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125056+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:03.125056+0000","last_peered":"2024-08-20T06:36:03.125056+0000","last_clean":"2024-08-20T06:36:03.125056+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:03.125056+0000","last_undegraded":"2024-08-20T06:36:03.125056+0000","last_fullsized":"2024-08-20T06:36:03.125056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124679+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:03.124679+0000","last_peered":"2024-08-20T06:36:03.124679+0000","last_clean":"2024-08-20T06:36:03.124679+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:03.124679+0000","last_undegraded":"2024-08-20T06:36:03.124679+0000","last_fullsized":"2024-08-20T06:36:03.124679+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126920+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:03.126920+0000","last_peered":"2024-08-20T06:36:03.126920+0000","last_clean":"2024-08-20T06:36:03.126920+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:03.126920+0000","last_undegraded":"2024-08-20T06:36:03.126920+0000","last_fullsized":"2024-08-20T06:36:03.126920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124319+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:03.124319+0000","last_peered":"2024-08-20T06:36:03.124319+0000","last_clean":"2024-08-20T06:36:03.124319+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:03.124319+0000","last_undegraded":"2024-08-20T06:36:03.124319+0000","last_fullsized":"2024-08-20T06:36:03.124319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129780+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:03.129780+0000","last_peered":"2024-08-20T06:36:03.129780+0000","last_clean":"2024-08-20T06:36:03.129780+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:03.129780+0000","last_undegraded":"2024-08-20T06:36:03.129780+0000","last_fullsized":"2024-08-20T06:36:03.129780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125319+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:03.125319+0000","last_peered":"2024-08-20T06:36:03.125319+0000","last_clean":"2024-08-20T06:36:03.125319+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:03.125319+0000","last_undegraded":"2024-08-20T06:36:03.125319+0000","last_fullsized":"2024-08-20T06:36:03.125319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122876+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:03.122876+0000","last_peered":"2024-08-20T06:36:03.122876+0000","last_clean":"2024-08-20T06:36:03.122876+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:03.122876+0000","last_undegraded":"2024-08-20T06:36:03.122876+0000","last_fullsized":"2024-08-20T06:36:03.122876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339442+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:03.339442+0000","last_peered":"2024-08-20T06:36:03.339442+0000","last_clean":"2024-08-20T06:36:03.339442+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:03.339442+0000","last_undegraded":"2024-08-20T06:36:03.339442+0000","last_fullsized":"2024-08-20T06:36:03.339442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227016+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:03.227016+0000","last_peered":"2024-08-20T06:36:03.227016+0000","last_clean":"2024-08-20T06:36:03.227016+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:03.227016+0000","last_undegraded":"2024-08-20T06:36:03.227016+0000","last_fullsized":"2024-08-20T06:36:03.227016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122863+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:03.122863+0000","last_peered":"2024-08-20T06:36:03.122863+0000","last_clean":"2024-08-20T06:36:03.122863+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:03.122863+0000","last_undegraded":"2024-08-20T06:36:03.122863+0000","last_fullsized":"2024-08-20T06:36:03.122863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122802+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:03.122802+0000","last_peered":"2024-08-20T06:36:03.122802+0000","last_clean":"2024-08-20T06:36:03.122802+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:03.122802+0000","last_undegraded":"2024-08-20T06:36:03.122802+0000","last_fullsized":"2024-08-20T06:36:03.122802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131241+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:03.131241+0000","last_peered":"2024-08-20T06:36:03.131241+0000","last_clean":"2024-08-20T06:36:03.131241+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:03.131241+0000","last_undegraded":"2024-08-20T06:36:03.131241+0000","last_fullsized":"2024-08-20T06:36:03.131241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125701+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:03.125701+0000","last_peered":"2024-08-20T06:36:03.125701+0000","last_clean":"2024-08-20T06:36:03.125701+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:03.125701+0000","last_undegraded":"2024-08-20T06:36:03.125701+0000","last_fullsized":"2024-08-20T06:36:03.125701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123606+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:03.123606+0000","last_peered":"2024-08-20T06:36:03.123606+0000","last_clean":"2024-08-20T06:36:03.123606+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:03.123606+0000","last_undegraded":"2024-08-20T06:36:03.123606+0000","last_fullsized":"2024-08-20T06:36:03.123606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122107+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:03.122107+0000","last_peered":"2024-08-20T06:36:03.122107+0000","last_clean":"2024-08-20T06:36:03.122107+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:03.122107+0000","last_undegraded":"2024-08-20T06:36:03.122107+0000","last_fullsized":"2024-08-20T06:36:03.122107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125244+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:03.125244+0000","last_peered":"2024-08-20T06:36:03.125244+0000","last_clean":"2024-08-20T06:36:03.125244+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:03.125244+0000","last_undegraded":"2024-08-20T06:36:03.125244+0000","last_fullsized":"2024-08-20T06:36:03.125244+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227050+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:03.227050+0000","last_peered":"2024-08-20T06:36:03.227050+0000","last_clean":"2024-08-20T06:36:03.227050+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:03.227050+0000","last_undegraded":"2024-08-20T06:36:03.227050+0000","last_fullsized":"2024-08-20T06:36:03.227050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129402+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:03.129402+0000","last_peered":"2024-08-20T06:36:03.129402+0000","last_clean":"2024-08-20T06:36:03.129402+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:03.129402+0000","last_undegraded":"2024-08-20T06:36:03.129402+0000","last_fullsized":"2024-08-20T06:36:03.129402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124326+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:03.124326+0000","last_peered":"2024-08-20T06:36:03.124326+0000","last_clean":"2024-08-20T06:36:03.124326+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:03.124326+0000","last_undegraded":"2024-08-20T06:36:03.124326+0000","last_fullsized":"2024-08-20T06:36:03.124326+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120983+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:03.120983+0000","last_peered":"2024-08-20T06:36:03.120983+0000","last_clean":"2024-08-20T06:36:03.120983+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:03.120983+0000","last_undegraded":"2024-08-20T06:36:03.120983+0000","last_fullsized":"2024-08-20T06:36:03.120983+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124556+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:03.124556+0000","last_peered":"2024-08-20T06:36:03.124556+0000","last_clean":"2024-08-20T06:36:03.124556+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:03.124556+0000","last_undegraded":"2024-08-20T06:36:03.124556+0000","last_fullsized":"2024-08-20T06:36:03.124556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122256+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:03.122256+0000","last_peered":"2024-08-20T06:36:03.122256+0000","last_clean":"2024-08-20T06:36:03.122256+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:03.122256+0000","last_undegraded":"2024-08-20T06:36:03.122256+0000","last_fullsized":"2024-08-20T06:36:03.122256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126240+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:03.126240+0000","last_peered":"2024-08-20T06:36:03.126240+0000","last_clean":"2024-08-20T06:36:03.126240+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:03.126240+0000","last_undegraded":"2024-08-20T06:36:03.126240+0000","last_fullsized":"2024-08-20T06:36:03.126240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227013+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:03.227013+0000","last_peered":"2024-08-20T06:36:03.227013+0000","last_clean":"2024-08-20T06:36:03.227013+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:03.227013+0000","last_undegraded":"2024-08-20T06:36:03.227013+0000","last_fullsized":"2024-08-20T06:36:03.227013+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123039+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:03.123039+0000","last_peered":"2024-08-20T06:36:03.123039+0000","last_clean":"2024-08-20T06:36:03.123039+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:03.123039+0000","last_undegraded":"2024-08-20T06:36:03.123039+0000","last_fullsized":"2024-08-20T06:36:03.123039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121952+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:03.121952+0000","last_peered":"2024-08-20T06:36:03.121952+0000","last_clean":"2024-08-20T06:36:03.121952+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:03.121952+0000","last_undegraded":"2024-08-20T06:36:03.121952+0000","last_fullsized":"2024-08-20T06:36:03.121952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122088+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:03.122088+0000","last_peered":"2024-08-20T06:36:03.122088+0000","last_clean":"2024-08-20T06:36:03.122088+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:03.122088+0000","last_undegraded":"2024-08-20T06:36:03.122088+0000","last_fullsized":"2024-08-20T06:36:03.122088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125069+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:03.125069+0000","last_peered":"2024-08-20T06:36:03.125069+0000","last_clean":"2024-08-20T06:36:03.125069+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:03.125069+0000","last_undegraded":"2024-08-20T06:36:03.125069+0000","last_fullsized":"2024-08-20T06:36:03.125069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124791+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:03.124791+0000","last_peered":"2024-08-20T06:36:03.124791+0000","last_clean":"2024-08-20T06:36:03.124791+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:03.124791+0000","last_undegraded":"2024-08-20T06:36:03.124791+0000","last_fullsized":"2024-08-20T06:36:03.124791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:09.073124+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:09.073124+0000","last_peered":"2024-08-20T06:36:09.073124+0000","last_clean":"2024-08-20T06:36:09.073124+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:09.073124+0000","last_undegraded":"2024-08-20T06:36:09.073124+0000","last_fullsized":"2024-08-20T06:36:09.073124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121272+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:03.121272+0000","last_peered":"2024-08-20T06:36:03.121272+0000","last_clean":"2024-08-20T06:36:03.121272+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:03.121272+0000","last_undegraded":"2024-08-20T06:36:03.121272+0000","last_fullsized":"2024-08-20T06:36:03.121272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226778+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:03.226778+0000","last_peered":"2024-08-20T06:36:03.226778+0000","last_clean":"2024-08-20T06:36:03.226778+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:03.226778+0000","last_undegraded":"2024-08-20T06:36:03.226778+0000","last_fullsized":"2024-08-20T06:36:03.226778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124413+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:03.124413+0000","last_peered":"2024-08-20T06:36:03.124413+0000","last_clean":"2024-08-20T06:36:03.124413+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:03.124413+0000","last_undegraded":"2024-08-20T06:36:03.124413+0000","last_fullsized":"2024-08-20T06:36:03.124413+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123444+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:03.123444+0000","last_peered":"2024-08-20T06:36:03.123444+0000","last_clean":"2024-08-20T06:36:03.123444+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:03.123444+0000","last_undegraded":"2024-08-20T06:36:03.123444+0000","last_fullsized":"2024-08-20T06:36:03.123444+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":84,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:06.067915+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:06.067915+0000","last_peered":"2024-08-20T06:36:06.067915+0000","last_clean":"2024-08-20T06:36:06.067915+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:06.067915+0000","last_undegraded":"2024-08-20T06:36:06.067915+0000","last_fullsized":"2024-08-20T06:36:06.067915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129663+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:03.129663+0000","last_peered":"2024-08-20T06:36:03.129663+0000","last_clean":"2024-08-20T06:36:03.129663+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:03.129663+0000","last_undegraded":"2024-08-20T06:36:03.129663+0000","last_fullsized":"2024-08-20T06:36:03.129663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.228019+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:03.228019+0000","last_peered":"2024-08-20T06:36:03.228019+0000","last_clean":"2024-08-20T06:36:03.228019+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:03.228019+0000","last_undegraded":"2024-08-20T06:36:03.228019+0000","last_fullsized":"2024-08-20T06:36:03.228019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129608+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:03.129608+0000","last_peered":"2024-08-20T06:36:03.129608+0000","last_clean":"2024-08-20T06:36:03.129608+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:03.129608+0000","last_undegraded":"2024-08-20T06:36:03.129608+0000","last_fullsized":"2024-08-20T06:36:03.129608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'29","reported_seq":193,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:06.348346+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:06.348346+0000","last_peered":"2024-08-20T06:36:06.348346+0000","last_clean":"2024-08-20T06:36:06.348346+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:06.348346+0000","last_undegraded":"2024-08-20T06:36:06.348346+0000","last_fullsized":"2024-08-20T06:36:06.348346+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126395+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:03.126395+0000","last_peered":"2024-08-20T06:36:03.126395+0000","last_clean":"2024-08-20T06:36:03.126395+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:03.126395+0000","last_undegraded":"2024-08-20T06:36:03.126395+0000","last_fullsized":"2024-08-20T06:36:03.126395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226852+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:03.226852+0000","last_peered":"2024-08-20T06:36:03.226852+0000","last_clean":"2024-08-20T06:36:03.226852+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:03.226852+0000","last_undegraded":"2024-08-20T06:36:03.226852+0000","last_fullsized":"2024-08-20T06:36:03.226852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227994+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:03.227994+0000","last_peered":"2024-08-20T06:36:03.227994+0000","last_clean":"2024-08-20T06:36:03.227994+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:03.227994+0000","last_undegraded":"2024-08-20T06:36:03.227994+0000","last_fullsized":"2024-08-20T06:36:03.227994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":87,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:07.069546+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:07.069546+0000","last_peered":"2024-08-20T06:36:07.069546+0000","last_clean":"2024-08-20T06:36:07.069546+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:07.069546+0000","last_undegraded":"2024-08-20T06:36:07.069546+0000","last_fullsized":"2024-08-20T06:36:07.069546+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":62,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121969+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:03.121969+0000","last_peered":"2024-08-20T06:36:03.121969+0000","last_clean":"2024-08-20T06:36:03.121969+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:03.121969+0000","last_undegraded":"2024-08-20T06:36:03.121969+0000","last_fullsized":"2024-08-20T06:36:03.121969+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120911+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:03.120911+0000","last_peered":"2024-08-20T06:36:03.120911+0000","last_clean":"2024-08-20T06:36:03.120911+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:03.120911+0000","last_undegraded":"2024-08-20T06:36:03.120911+0000","last_fullsized":"2024-08-20T06:36:03.120911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125364+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:03.125364+0000","last_peered":"2024-08-20T06:36:03.125364+0000","last_clean":"2024-08-20T06:36:03.125364+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:03.125364+0000","last_undegraded":"2024-08-20T06:36:03.125364+0000","last_fullsized":"2024-08-20T06:36:03.125364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227139+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:03.227139+0000","last_peered":"2024-08-20T06:36:03.227139+0000","last_clean":"2024-08-20T06:36:03.227139+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:03.227139+0000","last_undegraded":"2024-08-20T06:36:03.227139+0000","last_fullsized":"2024-08-20T06:36:03.227139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122921+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:03.122921+0000","last_peered":"2024-08-20T06:36:03.122921+0000","last_clean":"2024-08-20T06:36:03.122921+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:03.122921+0000","last_undegraded":"2024-08-20T06:36:03.122921+0000","last_fullsized":"2024-08-20T06:36:03.122921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.132123+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:03.132123+0000","last_peered":"2024-08-20T06:36:03.132123+0000","last_clean":"2024-08-20T06:36:03.132123+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:03.132123+0000","last_undegraded":"2024-08-20T06:36:03.132123+0000","last_fullsized":"2024-08-20T06:36:03.132123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123036+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:03.123036+0000","last_peered":"2024-08-20T06:36:03.123036+0000","last_clean":"2024-08-20T06:36:03.123036+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:03.123036+0000","last_undegraded":"2024-08-20T06:36:03.123036+0000","last_fullsized":"2024-08-20T06:36:03.123036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122598+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122598+0000","last_peered":"2024-08-20T06:36:03.122598+0000","last_clean":"2024-08-20T06:36:03.122598+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122598+0000","last_undegraded":"2024-08-20T06:36:03.122598+0000","last_fullsized":"2024-08-20T06:36:03.122598+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339216+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:03.339216+0000","last_peered":"2024-08-20T06:36:03.339216+0000","last_clean":"2024-08-20T06:36:03.339216+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:03.339216+0000","last_undegraded":"2024-08-20T06:36:03.339216+0000","last_fullsized":"2024-08-20T06:36:03.339216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124949+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:03.124949+0000","last_peered":"2024-08-20T06:36:03.124949+0000","last_clean":"2024-08-20T06:36:03.124949+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:03.124949+0000","last_undegraded":"2024-08-20T06:36:03.124949+0000","last_fullsized":"2024-08-20T06:36:03.124949+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.134693+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.134693+0000","last_peered":"2024-08-20T06:36:03.134693+0000","last_clean":"2024-08-20T06:36:03.134693+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.134693+0000","last_undegraded":"2024-08-20T06:36:03.134693+0000","last_fullsized":"2024-08-20T06:36:03.134693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226766+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:03.226766+0000","last_peered":"2024-08-20T06:36:03.226766+0000","last_clean":"2024-08-20T06:36:03.226766+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:03.226766+0000","last_undegraded":"2024-08-20T06:36:03.226766+0000","last_fullsized":"2024-08-20T06:36:03.226766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":85,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:06.067338+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:06.067338+0000","last_peered":"2024-08-20T06:36:06.067338+0000","last_clean":"2024-08-20T06:36:06.067338+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:06.067338+0000","last_undegraded":"2024-08-20T06:36:06.067338+0000","last_fullsized":"2024-08-20T06:36:06.067338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120920+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:03.120920+0000","last_peered":"2024-08-20T06:36:03.120920+0000","last_clean":"2024-08-20T06:36:03.120920+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:03.120920+0000","last_undegraded":"2024-08-20T06:36:03.120920+0000","last_fullsized":"2024-08-20T06:36:03.120920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125208+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:03.125208+0000","last_peered":"2024-08-20T06:36:03.125208+0000","last_clean":"2024-08-20T06:36:03.125208+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:03.125208+0000","last_undegraded":"2024-08-20T06:36:03.125208+0000","last_fullsized":"2024-08-20T06:36:03.125208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124981+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:03.124981+0000","last_peered":"2024-08-20T06:36:03.124981+0000","last_clean":"2024-08-20T06:36:03.124981+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:03.124981+0000","last_undegraded":"2024-08-20T06:36:03.124981+0000","last_fullsized":"2024-08-20T06:36:03.124981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227057+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:03.227057+0000","last_peered":"2024-08-20T06:36:03.227057+0000","last_clean":"2024-08-20T06:36:03.227057+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:03.227057+0000","last_undegraded":"2024-08-20T06:36:03.227057+0000","last_fullsized":"2024-08-20T06:36:03.227057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'10","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123630+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123630+0000","last_peered":"2024-08-20T06:36:03.123630+0000","last_clean":"2024-08-20T06:36:03.123630+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123630+0000","last_undegraded":"2024-08-20T06:36:03.123630+0000","last_fullsized":"2024-08-20T06:36:03.123630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124656+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:03.124656+0000","last_peered":"2024-08-20T06:36:03.124656+0000","last_clean":"2024-08-20T06:36:03.124656+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:03.124656+0000","last_undegraded":"2024-08-20T06:36:03.124656+0000","last_fullsized":"2024-08-20T06:36:03.124656+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339652+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:03.339652+0000","last_peered":"2024-08-20T06:36:03.339652+0000","last_clean":"2024-08-20T06:36:03.339652+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:03.339652+0000","last_undegraded":"2024-08-20T06:36:03.339652+0000","last_fullsized":"2024-08-20T06:36:03.339652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123213+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:03.123213+0000","last_peered":"2024-08-20T06:36:03.123213+0000","last_clean":"2024-08-20T06:36:03.123213+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:03.123213+0000","last_undegraded":"2024-08-20T06:36:03.123213+0000","last_fullsized":"2024-08-20T06:36:03.123213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124745+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:03.124745+0000","last_peered":"2024-08-20T06:36:03.124745+0000","last_clean":"2024-08-20T06:36:03.124745+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:03.124745+0000","last_undegraded":"2024-08-20T06:36:03.124745+0000","last_fullsized":"2024-08-20T06:36:03.124745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124496+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:03.124496+0000","last_peered":"2024-08-20T06:36:03.124496+0000","last_clean":"2024-08-20T06:36:03.124496+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:03.124496+0000","last_undegraded":"2024-08-20T06:36:03.124496+0000","last_fullsized":"2024-08-20T06:36:03.124496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125248+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:03.125248+0000","last_peered":"2024-08-20T06:36:03.125248+0000","last_clean":"2024-08-20T06:36:03.125248+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:03.125248+0000","last_undegraded":"2024-08-20T06:36:03.125248+0000","last_fullsized":"2024-08-20T06:36:03.125248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124605+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:03.124605+0000","last_peered":"2024-08-20T06:36:03.124605+0000","last_clean":"2024-08-20T06:36:03.124605+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:03.124605+0000","last_undegraded":"2024-08-20T06:36:03.124605+0000","last_fullsized":"2024-08-20T06:36:03.124605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122092+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:03.122092+0000","last_peered":"2024-08-20T06:36:03.122092+0000","last_clean":"2024-08-20T06:36:03.122092+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:03.122092+0000","last_undegraded":"2024-08-20T06:36:03.122092+0000","last_fullsized":"2024-08-20T06:36:03.122092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121464+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:03.121464+0000","last_peered":"2024-08-20T06:36:03.121464+0000","last_clean":"2024-08-20T06:36:03.121464+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:03.121464+0000","last_undegraded":"2024-08-20T06:36:03.121464+0000","last_fullsized":"2024-08-20T06:36:03.121464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129497+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:03.129497+0000","last_peered":"2024-08-20T06:36:03.129497+0000","last_clean":"2024-08-20T06:36:03.129497+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:03.129497+0000","last_undegraded":"2024-08-20T06:36:03.129497+0000","last_fullsized":"2024-08-20T06:36:03.129497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123408+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:03.123408+0000","last_peered":"2024-08-20T06:36:03.123408+0000","last_clean":"2024-08-20T06:36:03.123408+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:03.123408+0000","last_undegraded":"2024-08-20T06:36:03.123408+0000","last_fullsized":"2024-08-20T06:36:03.123408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125275+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:03.125275+0000","last_peered":"2024-08-20T06:36:03.125275+0000","last_clean":"2024-08-20T06:36:03.125275+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:03.125275+0000","last_undegraded":"2024-08-20T06:36:03.125275+0000","last_fullsized":"2024-08-20T06:36:03.125275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125182+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:03.125182+0000","last_peered":"2024-08-20T06:36:03.125182+0000","last_clean":"2024-08-20T06:36:03.125182+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:03.125182+0000","last_undegraded":"2024-08-20T06:36:03.125182+0000","last_fullsized":"2024-08-20T06:36:03.125182+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":373,"num_read_kb":253,"num_write":161,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037791,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1495040,"data_stored":1038910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168613,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528366,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1409024,"data_stored":977374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724594,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":802816,"data_stored":385304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888120,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1421312,"data_stored":977058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051645,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":447491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":815104,"data_stored":385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444103,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607625,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":446847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":823296,"data_stored":386575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:13 smithi183 ceph-mon[22204]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 6.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:13 smithi183 ceph-mon[22204]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:13 smithi193 ceph-mon[21944]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 6.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:13 smithi193 ceph-mon[21944]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:13 smithi107 ceph-mon[18495]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 6.6 KiB/s, 1 keys/s, 1 objects/s recovering 2024-08-20T06:36:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:13 smithi107 ceph-mon[18495]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:15.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:15 smithi183 ceph-mon[22204]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T06:36:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:15 smithi193 ceph-mon[21944]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T06:36:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:15 smithi107 ceph-mon[18495]: pgmap v309: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 6 op/s; 5.9 KiB/s, 0 keys/s, 1 objects/s recovering 2024-08-20T06:36:15.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T06:36:16.178 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 -- 172.21.15.107:0/612527336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc108e40 msgr2=0x7fabfc10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 --2- 172.21.15.107:0/612527336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc108e40 0x7fabfc10f640 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009ca0 tx=0x7fabf402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 -- 172.21.15.107:0/612527336 shutdown_connections 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 --2- 172.21.15.107:0/612527336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc108e40 0x7fabfc10f640 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 --2- 172.21.15.107:0/612527336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 0x7fabfc108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 --2- 172.21.15.107:0/612527336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc102ab0 0x7fabfc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 -- 172.21.15.107:0/612527336 >> 172.21.15.107:0/612527336 conn(0x7fabfc0fe500 msgr2=0x7fabfc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 -- 172.21.15.107:0/612527336 shutdown_connections 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.714+0000 7fac00f95640 1 -- 172.21.15.107:0/612527336 wait complete. 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 Processor -- start 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 -- start start 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 0x7fabfc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 0x7fabfc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112a00 con 0x7fabfc102ab0 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112880 con 0x7fabfc1084c0 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fac00f95640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112b80 con 0x7fabfc108e40 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fabfa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41152/0 (socket says 172.21.15.107:41152) 2024-08-20T06:36:16.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fabfa59c640 1 -- 172.21.15.107:0/1155160322 learned_addr learned my addr 172.21.15.107:0/1155160322 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fabfad9d640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 0x7fabfc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.715+0000 7fabf9d9b640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 0x7fabfc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 -- 172.21.15.107:0/1155160322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 msgr2=0x7fabfc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 0x7fabfc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 -- 172.21.15.107:0/1155160322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 msgr2=0x7fabfc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 0x7fabfc075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 -- 172.21.15.107:0/1155160322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1a7290 con 0x7fabfc102ab0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfad9d640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 0x7fabfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabfa59c640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c970 tx=0x7fabf000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0007bb0 con 0x7fabfc102ab0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fabf0007d50 con 0x7fabfc102ab0 2024-08-20T06:36:16.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc1a7580 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0004e40 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.716+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a7a70 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.717+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc102e90 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.718+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fabf0005000 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.719+0000 7fabeb7fe640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 0x7fabe40b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.719+0000 7fabf9d9b640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 0x7fabe40b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.719+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fabf00e3f90 con 0x7fabfc102ab0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.720+0000 7fabf9d9b640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 0x7fabe40b6120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fabec005fd0 tx=0x7fabec009470 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:16.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.722+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fabf0014030 con 0x7fabfc102ab0 2024-08-20T06:36:16.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.896+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabfc0632c0 con 0x7fabe40b3c60 2024-08-20T06:36:16.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.900+0000 7fabeb7fe640 1 -- 172.21.15.107:0/1155160322 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258730 (secure 0 0 0) 0x7fabfc0632c0 con 0x7fabe40b3c60 2024-08-20T06:36:16.903 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:16.904 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 msgr2=0x7fabe40b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 0x7fabe40b6120 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fabec005fd0 tx=0x7fabec009470 comp rx=0 tx=0).stop 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 msgr2=0x7fabfc0751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fabf000c970 tx=0x7fabf000ce30 comp rx=0 tx=0).stop 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 shutdown_connections 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fabe40b3c60 0x7fabe40b6120 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabfc108e40 0x7fabfc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fabfc1084c0 0x7fabfc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 --2- 172.21.15.107:0/1155160322 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fabfc102ab0 0x7fabfc0751d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:16.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 >> 172.21.15.107:0/1155160322 conn(0x7fabfc0fe500 msgr2=0x7fabfc1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:16.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 shutdown_connections 2024-08-20T06:36:16.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:16.903+0000 7fac00f95640 1 -- 172.21.15.107:0/1155160322 wait complete. 2024-08-20T06:36:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:17 smithi183 ceph-mon[22204]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:17.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:17 smithi107 ceph-mon[18495]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:17.539 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-08-20T06:36:15.323957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655962,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":479,"num_read_kb":332,"num_write":286,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369124,"kb_used_data":11484,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572700,"statfs":{"total":1151940427776,"available":1151562444800,"internally_reserved":0,"allocated":11759616,"data_stored":6589050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":28,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":61923,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125647+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:03.125647+0000","last_peered":"2024-08-20T06:36:03.125647+0000","last_clean":"2024-08-20T06:36:03.125647+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:03.125647+0000","last_undegraded":"2024-08-20T06:36:03.125647+0000","last_fullsized":"2024-08-20T06:36:03.125647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123128+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:03.123128+0000","last_peered":"2024-08-20T06:36:03.123128+0000","last_clean":"2024-08-20T06:36:03.123128+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:03.123128+0000","last_undegraded":"2024-08-20T06:36:03.123128+0000","last_fullsized":"2024-08-20T06:36:03.123128+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123174+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:03.123174+0000","last_peered":"2024-08-20T06:36:03.123174+0000","last_clean":"2024-08-20T06:36:03.123174+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:03.123174+0000","last_undegraded":"2024-08-20T06:36:03.123174+0000","last_fullsized":"2024-08-20T06:36:03.123174+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.127705+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:03.127705+0000","last_peered":"2024-08-20T06:36:03.127705+0000","last_clean":"2024-08-20T06:36:03.127705+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:03.127705+0000","last_undegraded":"2024-08-20T06:36:03.127705+0000","last_fullsized":"2024-08-20T06:36:03.127705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125056+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:03.125056+0000","last_peered":"2024-08-20T06:36:03.125056+0000","last_clean":"2024-08-20T06:36:03.125056+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:03.125056+0000","last_undegraded":"2024-08-20T06:36:03.125056+0000","last_fullsized":"2024-08-20T06:36:03.125056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124679+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:03.124679+0000","last_peered":"2024-08-20T06:36:03.124679+0000","last_clean":"2024-08-20T06:36:03.124679+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:03.124679+0000","last_undegraded":"2024-08-20T06:36:03.124679+0000","last_fullsized":"2024-08-20T06:36:03.124679+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126920+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:03.126920+0000","last_peered":"2024-08-20T06:36:03.126920+0000","last_clean":"2024-08-20T06:36:03.126920+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:03.126920+0000","last_undegraded":"2024-08-20T06:36:03.126920+0000","last_fullsized":"2024-08-20T06:36:03.126920+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124319+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:03.124319+0000","last_peered":"2024-08-20T06:36:03.124319+0000","last_clean":"2024-08-20T06:36:03.124319+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:03.124319+0000","last_undegraded":"2024-08-20T06:36:03.124319+0000","last_fullsized":"2024-08-20T06:36:03.124319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129780+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:03.129780+0000","last_peered":"2024-08-20T06:36:03.129780+0000","last_clean":"2024-08-20T06:36:03.129780+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:03.129780+0000","last_undegraded":"2024-08-20T06:36:03.129780+0000","last_fullsized":"2024-08-20T06:36:03.129780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125319+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:03.125319+0000","last_peered":"2024-08-20T06:36:03.125319+0000","last_clean":"2024-08-20T06:36:03.125319+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:03.125319+0000","last_undegraded":"2024-08-20T06:36:03.125319+0000","last_fullsized":"2024-08-20T06:36:03.125319+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122876+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:03.122876+0000","last_peered":"2024-08-20T06:36:03.122876+0000","last_clean":"2024-08-20T06:36:03.122876+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:03.122876+0000","last_undegraded":"2024-08-20T06:36:03.122876+0000","last_fullsized":"2024-08-20T06:36:03.122876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339442+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:03.339442+0000","last_peered":"2024-08-20T06:36:03.339442+0000","last_clean":"2024-08-20T06:36:03.339442+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:03.339442+0000","last_undegraded":"2024-08-20T06:36:03.339442+0000","last_fullsized":"2024-08-20T06:36:03.339442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227016+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:03.227016+0000","last_peered":"2024-08-20T06:36:03.227016+0000","last_clean":"2024-08-20T06:36:03.227016+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:03.227016+0000","last_undegraded":"2024-08-20T06:36:03.227016+0000","last_fullsized":"2024-08-20T06:36:03.227016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122863+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:03.122863+0000","last_peered":"2024-08-20T06:36:03.122863+0000","last_clean":"2024-08-20T06:36:03.122863+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:03.122863+0000","last_undegraded":"2024-08-20T06:36:03.122863+0000","last_fullsized":"2024-08-20T06:36:03.122863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122802+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:03.122802+0000","last_peered":"2024-08-20T06:36:03.122802+0000","last_clean":"2024-08-20T06:36:03.122802+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:03.122802+0000","last_undegraded":"2024-08-20T06:36:03.122802+0000","last_fullsized":"2024-08-20T06:36:03.122802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131241+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:03.131241+0000","last_peered":"2024-08-20T06:36:03.131241+0000","last_clean":"2024-08-20T06:36:03.131241+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:03.131241+0000","last_undegraded":"2024-08-20T06:36:03.131241+0000","last_fullsized":"2024-08-20T06:36:03.131241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125701+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:03.125701+0000","last_peered":"2024-08-20T06:36:03.125701+0000","last_clean":"2024-08-20T06:36:03.125701+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:03.125701+0000","last_undegraded":"2024-08-20T06:36:03.125701+0000","last_fullsized":"2024-08-20T06:36:03.125701+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123606+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:03.123606+0000","last_peered":"2024-08-20T06:36:03.123606+0000","last_clean":"2024-08-20T06:36:03.123606+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:03.123606+0000","last_undegraded":"2024-08-20T06:36:03.123606+0000","last_fullsized":"2024-08-20T06:36:03.123606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122107+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:03.122107+0000","last_peered":"2024-08-20T06:36:03.122107+0000","last_clean":"2024-08-20T06:36:03.122107+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:03.122107+0000","last_undegraded":"2024-08-20T06:36:03.122107+0000","last_fullsized":"2024-08-20T06:36:03.122107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125244+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:03.125244+0000","last_peered":"2024-08-20T06:36:03.125244+0000","last_clean":"2024-08-20T06:36:03.125244+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:03.125244+0000","last_undegraded":"2024-08-20T06:36:03.125244+0000","last_fullsized":"2024-08-20T06:36:03.125244+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227050+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:03.227050+0000","last_peered":"2024-08-20T06:36:03.227050+0000","last_clean":"2024-08-20T06:36:03.227050+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:03.227050+0000","last_undegraded":"2024-08-20T06:36:03.227050+0000","last_fullsized":"2024-08-20T06:36:03.227050+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129402+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:03.129402+0000","last_peered":"2024-08-20T06:36:03.129402+0000","last_clean":"2024-08-20T06:36:03.129402+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:03.129402+0000","last_undegraded":"2024-08-20T06:36:03.129402+0000","last_fullsized":"2024-08-20T06:36:03.129402+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124326+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:03.124326+0000","last_peered":"2024-08-20T06:36:03.124326+0000","last_clean":"2024-08-20T06:36:03.124326+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:03.124326+0000","last_undegraded":"2024-08-20T06:36:03.124326+0000","last_fullsized":"2024-08-20T06:36:03.124326+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120983+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:03.120983+0000","last_peered":"2024-08-20T06:36:03.120983+0000","last_clean":"2024-08-20T06:36:03.120983+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:03.120983+0000","last_undegraded":"2024-08-20T06:36:03.120983+0000","last_fullsized":"2024-08-20T06:36:03.120983+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124556+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:03.124556+0000","last_peered":"2024-08-20T06:36:03.124556+0000","last_clean":"2024-08-20T06:36:03.124556+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:03.124556+0000","last_undegraded":"2024-08-20T06:36:03.124556+0000","last_fullsized":"2024-08-20T06:36:03.124556+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122256+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:03.122256+0000","last_peered":"2024-08-20T06:36:03.122256+0000","last_clean":"2024-08-20T06:36:03.122256+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:03.122256+0000","last_undegraded":"2024-08-20T06:36:03.122256+0000","last_fullsized":"2024-08-20T06:36:03.122256+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126240+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:03.126240+0000","last_peered":"2024-08-20T06:36:03.126240+0000","last_clean":"2024-08-20T06:36:03.126240+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:03.126240+0000","last_undegraded":"2024-08-20T06:36:03.126240+0000","last_fullsized":"2024-08-20T06:36:03.126240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227013+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:03.227013+0000","last_peered":"2024-08-20T06:36:03.227013+0000","last_clean":"2024-08-20T06:36:03.227013+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:03.227013+0000","last_undegraded":"2024-08-20T06:36:03.227013+0000","last_fullsized":"2024-08-20T06:36:03.227013+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123039+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:03.123039+0000","last_peered":"2024-08-20T06:36:03.123039+0000","last_clean":"2024-08-20T06:36:03.123039+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:03.123039+0000","last_undegraded":"2024-08-20T06:36:03.123039+0000","last_fullsized":"2024-08-20T06:36:03.123039+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121952+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:03.121952+0000","last_peered":"2024-08-20T06:36:03.121952+0000","last_clean":"2024-08-20T06:36:03.121952+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:03.121952+0000","last_undegraded":"2024-08-20T06:36:03.121952+0000","last_fullsized":"2024-08-20T06:36:03.121952+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122088+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:03.122088+0000","last_peered":"2024-08-20T06:36:03.122088+0000","last_clean":"2024-08-20T06:36:03.122088+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:03.122088+0000","last_undegraded":"2024-08-20T06:36:03.122088+0000","last_fullsized":"2024-08-20T06:36:03.122088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125069+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:03.125069+0000","last_peered":"2024-08-20T06:36:03.125069+0000","last_clean":"2024-08-20T06:36:03.125069+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:03.125069+0000","last_undegraded":"2024-08-20T06:36:03.125069+0000","last_fullsized":"2024-08-20T06:36:03.125069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124791+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:03.124791+0000","last_peered":"2024-08-20T06:36:03.124791+0000","last_clean":"2024-08-20T06:36:03.124791+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:03.124791+0000","last_undegraded":"2024-08-20T06:36:03.124791+0000","last_fullsized":"2024-08-20T06:36:03.124791+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"71'5","reported_seq":96,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:14.081150+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:14.081150+0000","last_peered":"2024-08-20T06:36:14.081150+0000","last_clean":"2024-08-20T06:36:14.081150+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:14.081150+0000","last_undegraded":"2024-08-20T06:36:14.081150+0000","last_fullsized":"2024-08-20T06:36:14.081150+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":68,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121272+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:03.121272+0000","last_peered":"2024-08-20T06:36:03.121272+0000","last_clean":"2024-08-20T06:36:03.121272+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:03.121272+0000","last_undegraded":"2024-08-20T06:36:03.121272+0000","last_fullsized":"2024-08-20T06:36:03.121272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226778+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:03.226778+0000","last_peered":"2024-08-20T06:36:03.226778+0000","last_clean":"2024-08-20T06:36:03.226778+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:03.226778+0000","last_undegraded":"2024-08-20T06:36:03.226778+0000","last_fullsized":"2024-08-20T06:36:03.226778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124413+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:03.124413+0000","last_peered":"2024-08-20T06:36:03.124413+0000","last_clean":"2024-08-20T06:36:03.124413+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:03.124413+0000","last_undegraded":"2024-08-20T06:36:03.124413+0000","last_fullsized":"2024-08-20T06:36:03.124413+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123444+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:03.123444+0000","last_peered":"2024-08-20T06:36:03.123444+0000","last_clean":"2024-08-20T06:36:03.123444+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:03.123444+0000","last_undegraded":"2024-08-20T06:36:03.123444+0000","last_fullsized":"2024-08-20T06:36:03.123444+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"71'3","reported_seq":89,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:11.075679+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:11.075679+0000","last_peered":"2024-08-20T06:36:11.075679+0000","last_clean":"2024-08-20T06:36:11.075679+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:11.075679+0000","last_undegraded":"2024-08-20T06:36:11.075679+0000","last_fullsized":"2024-08-20T06:36:11.075679+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129663+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:03.129663+0000","last_peered":"2024-08-20T06:36:03.129663+0000","last_clean":"2024-08-20T06:36:03.129663+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:03.129663+0000","last_undegraded":"2024-08-20T06:36:03.129663+0000","last_fullsized":"2024-08-20T06:36:03.129663+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.228019+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:03.228019+0000","last_peered":"2024-08-20T06:36:03.228019+0000","last_clean":"2024-08-20T06:36:03.228019+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:03.228019+0000","last_undegraded":"2024-08-20T06:36:03.228019+0000","last_fullsized":"2024-08-20T06:36:03.228019+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129608+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:03.129608+0000","last_peered":"2024-08-20T06:36:03.129608+0000","last_clean":"2024-08-20T06:36:03.129608+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:03.129608+0000","last_undegraded":"2024-08-20T06:36:03.129608+0000","last_fullsized":"2024-08-20T06:36:03.129608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"71'29","reported_seq":203,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:11.356674+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:11.356674+0000","last_peered":"2024-08-20T06:36:11.356674+0000","last_clean":"2024-08-20T06:36:11.356674+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:11.356674+0000","last_undegraded":"2024-08-20T06:36:11.356674+0000","last_fullsized":"2024-08-20T06:36:11.356674+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61750,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":130,"num_read_kb":125,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126395+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:03.126395+0000","last_peered":"2024-08-20T06:36:03.126395+0000","last_clean":"2024-08-20T06:36:03.126395+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:03.126395+0000","last_undegraded":"2024-08-20T06:36:03.126395+0000","last_fullsized":"2024-08-20T06:36:03.126395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226852+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:03.226852+0000","last_peered":"2024-08-20T06:36:03.226852+0000","last_clean":"2024-08-20T06:36:03.226852+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:03.226852+0000","last_undegraded":"2024-08-20T06:36:03.226852+0000","last_fullsized":"2024-08-20T06:36:03.226852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227994+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:03.227994+0000","last_peered":"2024-08-20T06:36:03.227994+0000","last_clean":"2024-08-20T06:36:03.227994+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:03.227994+0000","last_undegraded":"2024-08-20T06:36:03.227994+0000","last_fullsized":"2024-08-20T06:36:03.227994+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"71'3","reported_seq":92,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:12.093351+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:12.093351+0000","last_peered":"2024-08-20T06:36:12.093351+0000","last_clean":"2024-08-20T06:36:12.093351+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:12.093351+0000","last_undegraded":"2024-08-20T06:36:12.093351+0000","last_fullsized":"2024-08-20T06:36:12.093351+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":67,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121969+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:03.121969+0000","last_peered":"2024-08-20T06:36:03.121969+0000","last_clean":"2024-08-20T06:36:03.121969+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:03.121969+0000","last_undegraded":"2024-08-20T06:36:03.121969+0000","last_fullsized":"2024-08-20T06:36:03.121969+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120911+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:03.120911+0000","last_peered":"2024-08-20T06:36:03.120911+0000","last_clean":"2024-08-20T06:36:03.120911+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:03.120911+0000","last_undegraded":"2024-08-20T06:36:03.120911+0000","last_fullsized":"2024-08-20T06:36:03.120911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125364+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:03.125364+0000","last_peered":"2024-08-20T06:36:03.125364+0000","last_clean":"2024-08-20T06:36:03.125364+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:03.125364+0000","last_undegraded":"2024-08-20T06:36:03.125364+0000","last_fullsized":"2024-08-20T06:36:03.125364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227139+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:03.227139+0000","last_peered":"2024-08-20T06:36:03.227139+0000","last_clean":"2024-08-20T06:36:03.227139+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:03.227139+0000","last_undegraded":"2024-08-20T06:36:03.227139+0000","last_fullsized":"2024-08-20T06:36:03.227139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122921+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:03.122921+0000","last_peered":"2024-08-20T06:36:03.122921+0000","last_clean":"2024-08-20T06:36:03.122921+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:03.122921+0000","last_undegraded":"2024-08-20T06:36:03.122921+0000","last_fullsized":"2024-08-20T06:36:03.122921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"71'1","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.132123+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:03.132123+0000","last_peered":"2024-08-20T06:36:03.132123+0000","last_clean":"2024-08-20T06:36:03.132123+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:03.132123+0000","last_undegraded":"2024-08-20T06:36:03.132123+0000","last_fullsized":"2024-08-20T06:36:03.132123+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123036+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:03.123036+0000","last_peered":"2024-08-20T06:36:03.123036+0000","last_clean":"2024-08-20T06:36:03.123036+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:03.123036+0000","last_undegraded":"2024-08-20T06:36:03.123036+0000","last_fullsized":"2024-08-20T06:36:03.123036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":91,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122598+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122598+0000","last_peered":"2024-08-20T06:36:03.122598+0000","last_clean":"2024-08-20T06:36:03.122598+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122598+0000","last_undegraded":"2024-08-20T06:36:03.122598+0000","last_fullsized":"2024-08-20T06:36:03.122598+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339216+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:03.339216+0000","last_peered":"2024-08-20T06:36:03.339216+0000","last_clean":"2024-08-20T06:36:03.339216+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:03.339216+0000","last_undegraded":"2024-08-20T06:36:03.339216+0000","last_fullsized":"2024-08-20T06:36:03.339216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124949+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:03.124949+0000","last_peered":"2024-08-20T06:36:03.124949+0000","last_clean":"2024-08-20T06:36:03.124949+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:03.124949+0000","last_undegraded":"2024-08-20T06:36:03.124949+0000","last_fullsized":"2024-08-20T06:36:03.124949+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":37,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.134693+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.134693+0000","last_peered":"2024-08-20T06:36:03.134693+0000","last_clean":"2024-08-20T06:36:03.134693+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.134693+0000","last_undegraded":"2024-08-20T06:36:03.134693+0000","last_fullsized":"2024-08-20T06:36:03.134693+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226766+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:03.226766+0000","last_peered":"2024-08-20T06:36:03.226766+0000","last_clean":"2024-08-20T06:36:03.226766+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:03.226766+0000","last_undegraded":"2024-08-20T06:36:03.226766+0000","last_fullsized":"2024-08-20T06:36:03.226766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"71'4","reported_seq":90,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:11.091254+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:11.091254+0000","last_peered":"2024-08-20T06:36:11.091254+0000","last_clean":"2024-08-20T06:36:11.091254+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:11.091254+0000","last_undegraded":"2024-08-20T06:36:11.091254+0000","last_fullsized":"2024-08-20T06:36:11.091254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":63,"num_read_kb":63,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.120920+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:03.120920+0000","last_peered":"2024-08-20T06:36:03.120920+0000","last_clean":"2024-08-20T06:36:03.120920+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:03.120920+0000","last_undegraded":"2024-08-20T06:36:03.120920+0000","last_fullsized":"2024-08-20T06:36:03.120920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125208+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:03.125208+0000","last_peered":"2024-08-20T06:36:03.125208+0000","last_clean":"2024-08-20T06:36:03.125208+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:03.125208+0000","last_undegraded":"2024-08-20T06:36:03.125208+0000","last_fullsized":"2024-08-20T06:36:03.125208+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"71'1","reported_seq":19,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124981+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:03.124981+0000","last_peered":"2024-08-20T06:36:03.124981+0000","last_clean":"2024-08-20T06:36:03.124981+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:03.124981+0000","last_undegraded":"2024-08-20T06:36:03.124981+0000","last_fullsized":"2024-08-20T06:36:03.124981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227057+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:03.227057+0000","last_peered":"2024-08-20T06:36:03.227057+0000","last_clean":"2024-08-20T06:36:03.227057+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:03.227057+0000","last_undegraded":"2024-08-20T06:36:03.227057+0000","last_fullsized":"2024-08-20T06:36:03.227057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"71'10","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123630+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123630+0000","last_peered":"2024-08-20T06:36:03.123630+0000","last_clean":"2024-08-20T06:36:03.123630+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123630+0000","last_undegraded":"2024-08-20T06:36:03.123630+0000","last_fullsized":"2024-08-20T06:36:03.123630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124656+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:03.124656+0000","last_peered":"2024-08-20T06:36:03.124656+0000","last_clean":"2024-08-20T06:36:03.124656+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:03.124656+0000","last_undegraded":"2024-08-20T06:36:03.124656+0000","last_fullsized":"2024-08-20T06:36:03.124656+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339652+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:03.339652+0000","last_peered":"2024-08-20T06:36:03.339652+0000","last_clean":"2024-08-20T06:36:03.339652+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:03.339652+0000","last_undegraded":"2024-08-20T06:36:03.339652+0000","last_fullsized":"2024-08-20T06:36:03.339652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123213+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:03.123213+0000","last_peered":"2024-08-20T06:36:03.123213+0000","last_clean":"2024-08-20T06:36:03.123213+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:03.123213+0000","last_undegraded":"2024-08-20T06:36:03.123213+0000","last_fullsized":"2024-08-20T06:36:03.123213+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124745+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:03.124745+0000","last_peered":"2024-08-20T06:36:03.124745+0000","last_clean":"2024-08-20T06:36:03.124745+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:03.124745+0000","last_undegraded":"2024-08-20T06:36:03.124745+0000","last_fullsized":"2024-08-20T06:36:03.124745+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124496+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:03.124496+0000","last_peered":"2024-08-20T06:36:03.124496+0000","last_clean":"2024-08-20T06:36:03.124496+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:03.124496+0000","last_undegraded":"2024-08-20T06:36:03.124496+0000","last_fullsized":"2024-08-20T06:36:03.124496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125248+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:03.125248+0000","last_peered":"2024-08-20T06:36:03.125248+0000","last_clean":"2024-08-20T06:36:03.125248+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:03.125248+0000","last_undegraded":"2024-08-20T06:36:03.125248+0000","last_fullsized":"2024-08-20T06:36:03.125248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124605+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:03.124605+0000","last_peered":"2024-08-20T06:36:03.124605+0000","last_clean":"2024-08-20T06:36:03.124605+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:03.124605+0000","last_undegraded":"2024-08-20T06:36:03.124605+0000","last_fullsized":"2024-08-20T06:36:03.124605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122092+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:03.122092+0000","last_peered":"2024-08-20T06:36:03.122092+0000","last_clean":"2024-08-20T06:36:03.122092+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:03.122092+0000","last_undegraded":"2024-08-20T06:36:03.122092+0000","last_fullsized":"2024-08-20T06:36:03.122092+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121464+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:03.121464+0000","last_peered":"2024-08-20T06:36:03.121464+0000","last_clean":"2024-08-20T06:36:03.121464+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:03.121464+0000","last_undegraded":"2024-08-20T06:36:03.121464+0000","last_fullsized":"2024-08-20T06:36:03.121464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.129497+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:03.129497+0000","last_peered":"2024-08-20T06:36:03.129497+0000","last_clean":"2024-08-20T06:36:03.129497+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:03.129497+0000","last_undegraded":"2024-08-20T06:36:03.129497+0000","last_fullsized":"2024-08-20T06:36:03.129497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123408+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:03.123408+0000","last_peered":"2024-08-20T06:36:03.123408+0000","last_clean":"2024-08-20T06:36:03.123408+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:03.123408+0000","last_undegraded":"2024-08-20T06:36:03.123408+0000","last_fullsized":"2024-08-20T06:36:03.123408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125275+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:03.125275+0000","last_peered":"2024-08-20T06:36:03.125275+0000","last_clean":"2024-08-20T06:36:03.125275+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:03.125275+0000","last_undegraded":"2024-08-20T06:36:03.125275+0000","last_fullsized":"2024-08-20T06:36:03.125275+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125182+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:03.125182+0000","last_peered":"2024-08-20T06:36:03.125182+0000","last_clean":"2024-08-20T06:36:03.125182+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:03.125182+0000","last_undegraded":"2024-08-20T06:36:03.125182+0000","last_fullsized":"2024-08-20T06:36:03.125182+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65360,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":403,"num_read_kb":268,"num_write":161,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037792,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28488,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716664,"statfs":{"total":95995035648,"available":95965863936,"internally_reserved":0,"allocated":1495040,"data_stored":1038910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168613,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36152,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709000,"statfs":{"total":95995035648,"available":95958016000,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31984,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713168,"statfs":{"total":95995035648,"available":95962284032,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528367,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32568,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712584,"statfs":{"total":95995035648,"available":95961686016,"internally_reserved":0,"allocated":1409024,"data_stored":977374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724595,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":802816,"data_stored":385304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888121,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28420,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716732,"statfs":{"total":95995035648,"available":95965933568,"internally_reserved":0,"allocated":1421312,"data_stored":977058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051646,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32044,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713108,"statfs":{"total":95995035648,"available":95962222592,"internally_reserved":0,"allocated":876544,"data_stored":447491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":815104,"data_stored":385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607626,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":446847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":823296,"data_stored":386575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:17.543 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T06:36:17.543 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-20T06:36:17.543 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-20T06:36:17.544 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-20T06:36:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:17 smithi193 ceph-mon[21944]: pgmap v310: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:17.791 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:18.214 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:18 smithi107 ceph-mon[18495]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:18.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 -- 172.21.15.107:0/1599441375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 msgr2=0x7f2568107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 --2- 172.21.15.107:0/1599441375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f2568107800 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2550009900 tx=0x7f255002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 -- 172.21.15.107:0/1599441375 shutdown_connections 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 --2- 172.21.15.107:0/1599441375 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2568107d40 0x7f256810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 --2- 172.21.15.107:0/1599441375 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f2568107800 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 --2- 172.21.15.107:0/1599441375 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25681019b0 0x7f2568101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 -- 172.21.15.107:0/1599441375 >> 172.21.15.107:0/1599441375 conn(0x7f25680fd400 msgr2=0x7f25680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 -- 172.21.15.107:0/1599441375 shutdown_connections 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 -- 172.21.15.107:0/1599441375 wait complete. 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.292+0000 7f256d77e640 1 Processor -- start 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- start start 2024-08-20T06:36:18.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681019b0 0x7f256810f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2568107d40 0x7f256810fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f25681129f0 con 0x7f25681073c0 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2568112870 con 0x7f25681019b0 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2568112b70 con 0x7f2568107d40 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41178/0 (socket says 172.21.15.107:41178) 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 -- 172.21.15.107:0/4198996317 learned_addr learned my addr 172.21.15.107:0/4198996317 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f2566ffd640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681019b0 0x7f256810f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 -- 172.21.15.107:0/4198996317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2568107d40 msgr2=0x7f256810fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2568107d40 0x7f256810fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 -- 172.21.15.107:0/4198996317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681019b0 msgr2=0x7f256810f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:18.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681019b0 0x7f256810f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 -- 172.21.15.107:0/4198996317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25681a74c0 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f255ffff640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2550009a20 tx=0x7f2550004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255003f030 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f25500035f0 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25681a7750 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550035070 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.293+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25681a7c30 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.294+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2568101e10 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.296+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2550003aa0 con 0x7f25681073c0 2024-08-20T06:36:18.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.296+0000 7f2564ff9640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 0x7f25540b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:18.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.297+0000 7f2566ffd640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 0x7f25540b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:18.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.297+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2550104dd0 con 0x7f25681073c0 2024-08-20T06:36:18.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.297+0000 7f2566ffd640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 0x7f25540b6120 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2558009890 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:18.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.299+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f25500cc2c0 con 0x7f25681073c0 2024-08-20T06:36:18.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:18 smithi183 ceph-mon[22204]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:18.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.512+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2568110350 con 0x7f25681073c0 2024-08-20T06:36:18.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.512+0000 7f2564ff9640 1 -- 172.21.15.107:0/4198996317 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2568110350 con 0x7f25681073c0 2024-08-20T06:36:18.513 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:18.513 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-20T06:36:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 msgr2=0x7f25540b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 0x7f25540b6120 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2558009890 tx=0x7f2558008040 comp rx=0 tx=0).stop 2024-08-20T06:36:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 msgr2=0x7f256810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2550009a20 tx=0x7f2550004270 comp rx=0 tx=0).stop 2024-08-20T06:36:18.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 shutdown_connections 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f25540b3c60 0x7f25540b6120 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2568107d40 0x7f256810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25681073c0 0x7f256810f590 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 --2- 172.21.15.107:0/4198996317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f25681019b0 0x7f256810f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 >> 172.21.15.107:0/4198996317 conn(0x7f25680fd400 msgr2=0x7f25680ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 shutdown_connections 2024-08-20T06:36:18.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:18.514+0000 7f256d77e640 1 -- 172.21.15.107:0/4198996317 wait complete. 2024-08-20T06:36:18.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:18 smithi193 ceph-mon[21944]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:36:19.095 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-20T06:36:19.096 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:19.344 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:19 smithi107 ceph-mon[18495]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:19 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4198996317' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T06:36:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:19 smithi183 ceph-mon[22204]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:19 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4198996317' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T06:36:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:19 smithi193 ceph-mon[21944]: pgmap v311: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 5 op/s; 5.0 KiB/s, 0 keys/s, 0 objects/s recovering 2024-08-20T06:36:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:19 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4198996317' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 -- 172.21.15.107:0/1660876234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 msgr2=0x7f2be4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 --2- 172.21.15.107:0/1660876234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4108920 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009ca0 tx=0x7f2bd802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 -- 172.21.15.107:0/1660876234 shutdown_connections 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 --2- 172.21.15.107:0/1660876234 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4108e60 0x7f2be410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 --2- 172.21.15.107:0/1660876234 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4108920 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 --2- 172.21.15.107:0/1660876234 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4102ad0 0x7f2be4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 -- 172.21.15.107:0/1660876234 >> 172.21.15.107:0/1660876234 conn(0x7f2be40fe540 msgr2=0x7f2be4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 -- 172.21.15.107:0/1660876234 shutdown_connections 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.840+0000 7f2bebc65640 1 -- 172.21.15.107:0/1660876234 wait complete. 2024-08-20T06:36:19.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 Processor -- start 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 -- start start 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 0x7f2be4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 0x7f2be419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2be41129e0 con 0x7f2be41084e0 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4112860 con 0x7f2be4108e60 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2bebc65640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2be4112b60 con 0x7f2be4102ad0 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2be9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2be9200640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41198/0 (socket says 172.21.15.107:41198) 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2be9200640 1 -- 172.21.15.107:0/776509008 learned_addr learned my addr 172.21.15.107:0/776509008 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.841+0000 7f2be9a01640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 0x7f2be4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2bea202640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 0x7f2be419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:19.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 -- 172.21.15.107:0/776509008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 msgr2=0x7f2be4198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 0x7f2be4198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 -- 172.21.15.107:0/776509008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 msgr2=0x7f2be419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 0x7f2be419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 -- 172.21.15.107:0/776509008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be419efa0 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9a01640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 0x7f2be4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2bea202640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 0x7f2be419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be9200640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006010 tx=0x7f2bd8032bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8004040 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2bd8004670 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be419f230 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8030da0 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.842+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be41a5950 con 0x7f2be41084e0 2024-08-20T06:36:19.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.843+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be4102eb0 con 0x7f2be41084e0 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.844+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2bd80305c0 con 0x7f2be41084e0 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.845+0000 7f2be2ffd640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 0x7f2bcc0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.845+0000 7f2be9a01640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 0x7f2bcc0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.846+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2bd8105600 con 0x7f2be41084e0 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.846+0000 7f2be9a01640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 0x7f2bcc0b6120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0058c0 tx=0x7f2bdc005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:19.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:19.849+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2bd8004820 con 0x7f2be41084e0 2024-08-20T06:36:20.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.062+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2be419a020 con 0x7f2be41084e0 2024-08-20T06:36:20.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.063+0000 7f2be2ffd640 1 -- 172.21.15.107:0/776509008 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f2bd80ccbc0 con 0x7f2be41084e0 2024-08-20T06:36:20.064 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:20.064 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:36:20.064 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:20.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 msgr2=0x7f2bcc0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:20.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 0x7f2bcc0b6120 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0058c0 tx=0x7f2bdc005d30 comp rx=0 tx=0).stop 2024-08-20T06:36:20.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 msgr2=0x7f2be4199430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8006010 tx=0x7f2bd8032bc0 comp rx=0 tx=0).stop 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 shutdown_connections 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2be4108e60 0x7f2be419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2be41084e0 0x7f2be4199430 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2bcc0b3c60 0x7f2bcc0b6120 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 --2- 172.21.15.107:0/776509008 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be4102ad0 0x7f2be4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 >> 172.21.15.107:0/776509008 conn(0x7f2be40fe540 msgr2=0x7f2be4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 shutdown_connections 2024-08-20T06:36:20.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:20.065+0000 7f2bebc65640 1 -- 172.21.15.107:0/776509008 wait complete. 2024-08-20T06:36:20.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:20 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/776509008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:20.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:20 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/776509008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:20.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:20 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/776509008' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:20.721 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:20.975 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:21 smithi107 ceph-mon[18495]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:21 smithi183 ceph-mon[22204]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 -- 172.21.15.107:0/3827478893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224108e60 msgr2=0x7fd22410f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/3827478893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224108e60 0x7fd22410f660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009ca0 tx=0x7fd21c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 -- 172.21.15.107:0/3827478893 shutdown_connections 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/3827478893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224108e60 0x7fd22410f660 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/3827478893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2241084e0 0x7fd224108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/3827478893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd224102ad0 0x7fd224102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 -- 172.21.15.107:0/3827478893 >> 172.21.15.107:0/3827478893 conn(0x7fd2240fe540 msgr2=0x7fd224100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 -- 172.21.15.107:0/3827478893 shutdown_connections 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.463+0000 7fd22bb8b640 1 -- 172.21.15.107:0/3827478893 wait complete. 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 Processor -- start 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 -- start start 2024-08-20T06:36:21.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2241084e0 0x7fd2241994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 0x7fd22419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:21.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd224112a60 con 0x7fd224102ad0 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2241128e0 con 0x7fd224108e60 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22bb8b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd224112be0 con 0x7fd2241084e0 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41208/0 (socket says 172.21.15.107:41208) 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 -- 172.21.15.107:0/204250463 learned_addr learned my addr 172.21.15.107:0/204250463 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229126640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2241084e0 0x7fd2241994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22a128640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 0x7fd22419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 -- 172.21.15.107:0/204250463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2241084e0 msgr2=0x7fd2241994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2241084e0 0x7fd2241994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 -- 172.21.15.107:0/204250463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 msgr2=0x7fd22419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 0x7fd22419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd229927640 1 -- 172.21.15.107:0/204250463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22419f020 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.464+0000 7fd22a128640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 0x7fd22419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd229927640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b4a0 tx=0x7fd22000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220015020 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd220011cc0 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd220010d00 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22419f310 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.465+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2241a5ba0 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.466+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd224102eb0 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.467+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fd220018020 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.468+0000 7fd21affd640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 0x7fd2100b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.468+0000 7fd229126640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 0x7fd2100b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.468+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd2200e3aa0 con 0x7fd224102ad0 2024-08-20T06:36:21.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.468+0000 7fd229126640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 0x7fd2100b61a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2241071e0 tx=0x7fd214009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:21.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.471+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd22001f020 con 0x7fd224102ad0 2024-08-20T06:36:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:21 smithi193 ceph-mon[21944]: pgmap v312: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:21.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.683+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd22419a0a0 con 0x7fd224102ad0 2024-08-20T06:36:21.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.683+0000 7fd21affd640 1 -- 172.21.15.107:0/204250463 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7fd2200ab0a0 con 0x7fd224102ad0 2024-08-20T06:36:21.685 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:21.685 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:36:21.685 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:21.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.685+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 msgr2=0x7fd2100b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.685+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 0x7fd2100b61a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd2241071e0 tx=0x7fd214009110 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.685+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 msgr2=0x7fd224198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd22000b4a0 tx=0x7fd22000b8a0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 shutdown_connections 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd224108e60 0x7fd22419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2100b3ce0 0x7fd2100b61a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2241084e0 0x7fd2241994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 --2- 172.21.15.107:0/204250463 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd224102ad0 0x7fd224198f70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 >> 172.21.15.107:0/204250463 conn(0x7fd2240fe540 msgr2=0x7fd224100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 shutdown_connections 2024-08-20T06:36:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:21.686+0000 7fd22bb8b640 1 -- 172.21.15.107:0/204250463 wait complete. 2024-08-20T06:36:22.346 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:36:22.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:22 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/204250463' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:22.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:22 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/204250463' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:22.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:22 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/204250463' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:22.589 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.025+0000 7f93df6c9640 1 -- 172.21.15.107:0/1236820832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 msgr2=0x7f93d810e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.025+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1236820832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d810e560 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009ca0 tx=0x7f93d002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 -- 172.21.15.107:0/1236820832 shutdown_connections 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1236820832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d810e560 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1236820832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 0x7f93d8107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1236820832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 -- 172.21.15.107:0/1236820832 >> 172.21.15.107:0/1236820832 conn(0x7f93d80fd440 msgr2=0x7f93d80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 -- 172.21.15.107:0/1236820832 shutdown_connections 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 -- 172.21.15.107:0/1236820832 wait complete. 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 Processor -- start 2024-08-20T06:36:23.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 -- start start 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.026+0000 7f93df6c9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 0x7f93d8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8111920 con 0x7f93d8107d60 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f93d81117a0 con 0x7f93d81073e0 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8111aa0 con 0x7f93d81019d0 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41230/0 (socket says 172.21.15.107:41230) 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 -- 172.21.15.107:0/1561772802 learned_addr learned my addr 172.21.15.107:0/1561772802 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93dcc64640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 0x7f93d8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93dd465640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:23.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 -- 172.21.15.107:0/1561772802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 msgr2=0x7f93d8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 -- 172.21.15.107:0/1561772802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 msgr2=0x7f93d8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 0x7f93d8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 -- 172.21.15.107:0/1561772802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d819ee90 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93dd465640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ddc66640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005cc0 tx=0x7f93d0032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0032e00 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f93d00044d0 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d819f120 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0030b70 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.027+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81a5840 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.028+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93d8101db0 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.030+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f93d0030d10 con 0x7f93d8107d60 2024-08-20T06:36:23.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.031+0000 7f93ca7fc640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 0x7f93c40b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.031+0000 7f93dd465640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 0x7f93c40b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.031+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f93d0105300 con 0x7f93d8107d60 2024-08-20T06:36:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.031+0000 7f93dd465640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 0x7f93c40b61a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009b40 tx=0x7f93d4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:23.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.034+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f93d00cc8c0 con 0x7f93d8107d60 2024-08-20T06:36:23.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.213+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93d8199f10 con 0x7f93d8107d60 2024-08-20T06:36:23.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.214+0000 7f93ca7fc640 1 -- 172.21.15.107:0/1561772802 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7f93d8199f10 con 0x7f93d8107d60 2024-08-20T06:36:23.216 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:23.216 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 msgr2=0x7f93c40b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 0x7f93c40b61a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009b40 tx=0x7f93d4005e50 comp rx=0 tx=0).stop 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 msgr2=0x7f93d819e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f93d0005cc0 tx=0x7f93d0032a10 comp rx=0 tx=0).stop 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 shutdown_connections 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f93d8107d60 0x7f93d819e6c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f93d81073e0 0x7f93d8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f93c40b3ce0 0x7f93c40b61a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 --2- 172.21.15.107:0/1561772802 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93d81019d0 0x7f93d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 >> 172.21.15.107:0/1561772802 conn(0x7f93d80fd440 msgr2=0x7f93d806b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 shutdown_connections 2024-08-20T06:36:23.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:23.217+0000 7f93df6c9640 1 -- 172.21.15.107:0/1561772802 wait complete. 2024-08-20T06:36:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:23 smithi107 ceph-mon[18495]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:23 smithi183 ceph-mon[22204]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:23 smithi193 ceph-mon[21944]: pgmap v313: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:23.897 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:24.189 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:24.310 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:24 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1561772802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:24.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:24 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1561772802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:24.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:24 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1561772802' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 -- 172.21.15.107:0/2360598314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc1084c0 msgr2=0x7f15cc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 --2- 172.21.15.107:0/2360598314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc1084c0 0x7f15cc108900 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009900 tx=0x7f15b402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 -- 172.21.15.107:0/2360598314 shutdown_connections 2024-08-20T06:36:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 --2- 172.21.15.107:0/2360598314 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc108e40 0x7f15cc10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 --2- 172.21.15.107:0/2360598314 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc1084c0 0x7f15cc108900 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 --2- 172.21.15.107:0/2360598314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc102ab0 0x7f15cc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 -- 172.21.15.107:0/2360598314 >> 172.21.15.107:0/2360598314 conn(0x7f15cc0fe500 msgr2=0x7f15cc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 -- 172.21.15.107:0/2360598314 shutdown_connections 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.692+0000 7f15d1538640 1 -- 172.21.15.107:0/2360598314 wait complete. 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 Processor -- start 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 -- start start 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc102ab0 0x7f15cc10e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc1084c0 0x7f15cc10ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc1129b0 con 0x7f15cc108e40 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc112830 con 0x7f15cc102ab0 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15d1538640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc112b30 con 0x7f15cc1084c0 2024-08-20T06:36:24.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15cb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41250/0 (socket says 172.21.15.107:41250) 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15cb7fe640 1 -- 172.21.15.107:0/2816132054 learned_addr learned my addr 172.21.15.107:0/2816132054 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15ca7fc640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc1084c0 0x7f15cc10ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15caffd640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc102ab0 0x7f15cc10e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.693+0000 7f15cb7fe640 1 -- 172.21.15.107:0/2816132054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc1084c0 msgr2=0x7f15cc10ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15cb7fe640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc1084c0 0x7f15cc10ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15cb7fe640 1 -- 172.21.15.107:0/2816132054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc102ab0 msgr2=0x7f15cc10e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15cb7fe640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc102ab0 0x7f15cc10e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15cb7fe640 1 -- 172.21.15.107:0/2816132054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc1a5550 con 0x7f15cc108e40 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15cb7fe640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15c400eaa0 tx=0x7f15c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c400cdb0 con 0x7f15cc108e40 2024-08-20T06:36:24.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f15c4005020 con 0x7f15cc108e40 2024-08-20T06:36:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15cc1a5840 con 0x7f15cc108e40 2024-08-20T06:36:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15c40116d0 con 0x7f15cc108e40 2024-08-20T06:36:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.694+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15cc1abf00 con 0x7f15cc108e40 2024-08-20T06:36:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.695+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15cc102e90 con 0x7f15cc108e40 2024-08-20T06:36:24.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.697+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f15c40051d0 con 0x7f15cc108e40 2024-08-20T06:36:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.698+0000 7f15c3fff640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 0x7f15b80b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.698+0000 7f15caffd640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 0x7f15b80b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.698+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f15c40e42d0 con 0x7f15cc108e40 2024-08-20T06:36:24.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.698+0000 7f15caffd640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 0x7f15b80b61a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:24.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.701+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f15c4016030 con 0x7f15cc108e40 2024-08-20T06:36:24.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.910+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15cc0ffff0 con 0x7f15cc108e40 2024-08-20T06:36:24.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.911+0000 7f15c3fff640 1 -- 172.21.15.107:0/2816132054 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f15cc0ffff0 con 0x7f15cc108e40 2024-08-20T06:36:24.912 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:24.912 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:36:24.912 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 msgr2=0x7f15b80b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 0x7f15b80b61a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005fd0 tx=0x7f15bc005e20 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 msgr2=0x7f15cc1a4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f15c400eaa0 tx=0x7f15c400ef60 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 shutdown_connections 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f15cc108e40 0x7f15cc1a4d80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f15b80b3ce0 0x7f15b80b61a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc1084c0 0x7f15cc10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 --2- 172.21.15.107:0/2816132054 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f15cc102ab0 0x7f15cc10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 >> 172.21.15.107:0/2816132054 conn(0x7f15cc0fe500 msgr2=0x7f15cc104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:24.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 shutdown_connections 2024-08-20T06:36:24.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:24.914+0000 7f15d1538640 1 -- 172.21.15.107:0/2816132054 wait complete. 2024-08-20T06:36:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:25 smithi183 ceph-mon[22204]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:36:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:25 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2816132054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:25.500 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:25 smithi107 ceph-mon[18495]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:36:25.500 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:25 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2816132054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:25.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T06:35:00.478558+0000', 'modified': '2024-08-20T06:35:11.548444+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24562, 'mds_2': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24533': {'gid': 24533, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2773184516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2773184516}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2773184516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24541': {'gid': 24541, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/2974563037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2974563037}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2974563037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/2519256232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2519256232}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2519256232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6837/2933033317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 2933033317}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 2933033317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/3581594961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3581594961}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3581594961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/2078682144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 2078682144}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 2078682144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24572, 'qdb_cluster': [24548, 24562, 24572]} 2024-08-20T06:36:25.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-20T06:36:25.529 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T06:36:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:25 smithi193 ceph-mon[21944]: pgmap v314: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:36:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:25 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2816132054' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:25.775 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 -- 172.21.15.107:0/2453274357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01019d0 msgr2=0x7fdfb0101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 --2- 172.21.15.107:0/2453274357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01019d0 0x7fdfb0101db0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009900 tx=0x7fdfac02a410 comp rx=0 tx=0).stop 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 -- 172.21.15.107:0/2453274357 shutdown_connections 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 --2- 172.21.15.107:0/2453274357 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb0107d60 0x7fdfb010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 --2- 172.21.15.107:0/2453274357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01073e0 0x7fdfb0107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 --2- 172.21.15.107:0/2453274357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb01019d0 0x7fdfb0101db0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 -- 172.21.15.107:0/2453274357 >> 172.21.15.107:0/2453274357 conn(0x7fdfb00fd440 msgr2=0x7fdfb00ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 -- 172.21.15.107:0/2453274357 shutdown_connections 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.284+0000 7fdfb781b640 1 -- 172.21.15.107:0/2453274357 wait complete. 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 Processor -- start 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 -- start start 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01019d0 0x7fdfb0198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 0x7fdfb0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01129f0 con 0x7fdfb0107d60 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0112870 con 0x7fdfb01073e0 2024-08-20T06:36:26.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb781b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0112b70 con 0x7fdfb01019d0 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb5db8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb5db8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58712/0 (socket says 172.21.15.107:58712) 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.285+0000 7fdfb5db8640 1 -- 172.21.15.107:0/3622766278 learned_addr learned my addr 172.21.15.107:0/3622766278 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01019d0 0x7fdfb0198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 0x7fdfb0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 -- 172.21.15.107:0/3622766278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01019d0 msgr2=0x7fdfb0198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01019d0 0x7fdfb0198f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 -- 172.21.15.107:0/3622766278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 msgr2=0x7fdfb0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 0x7fdfb0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 -- 172.21.15.107:0/3622766278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb019eff0 con 0x7fdfb0107d60 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 0x7fdfb0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:26.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb5db8640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800cad0 tx=0x7fdfa800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfa67fc640 1 -- 172.21.15.107:0/3622766278 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800ae20 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfa67fc640 1 -- 172.21.15.107:0/3622766278 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfa8005020 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb019f2e0 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfa67fc640 1 -- 172.21.15.107:0/3622766278 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800f6b0 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.286+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01a5a20 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.289+0000 7fdfb9c14740 1 Processor -- start 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.289+0000 7fdfa67fc640 1 -- 172.21.15.107:0/3622766278 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7fdfa80051d0 con 0x7fdfb0107d60 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 -- start start 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb5a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651bdac26a0 con 0x5651bdb4c510 2024-08-20T06:36:26.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651bdab1ba0 con 0x5651bdb4bbf0 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651bdab5be0 con 0x5651bdb584a0 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58726/0 (socket says 172.21.15.107:58726) 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 -- 172.21.15.107:0/3638120931 learned_addr learned my addr 172.21.15.107:0/3638120931 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb5a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfa67fc640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 0x7fdf9c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 -- 172.21.15.107:0/3638120931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 msgr2=0x5651bdb5a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb5a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 -- 172.21.15.107:0/3638120931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 msgr2=0x5651bdb4bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb4bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb55b7640 1 -- 172.21.15.107:0/3638120931 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bd811540 con 0x5651bdb4c510 2024-08-20T06:36:26.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb4bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfb5db8640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 0x7fdf9c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.290+0000 7fdfa67fc640 1 -- 172.21.15.107:0/3622766278 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdfa80e4130 con 0x7fdfb0107d60 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02a930 tx=0x7fdfac031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a619dfdd411649b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdf9b7fe640 1 -- 172.21.15.107:0/3638120931 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac035030 con 0x5651bdb4c510 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdf9b7fe640 1 -- 172.21.15.107:0/3638120931 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfac040070 con 0x5651bdb4c510 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdf9b7fe640 1 -- 172.21.15.107:0/3638120931 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003a80 con 0x5651bdb4c510 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3638120931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 msgr2=0x5651bdb57f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdfb5db8640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 0x7fdf9c0b6120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800caa0 tx=0x7fdfa800a890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.291+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02a930 tx=0x7fdfac031dc0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3638120931 shutdown_connections 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb5a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb57f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3638120931 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb4bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3638120931 >> 172.21.15.107:0/3638120931 conn(0x5651bdb49a20 msgr2=0x5651bdb4abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3638120931 shutdown_connections 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3638120931 wait complete. 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- start start 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 Processor -- start 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb55f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651bd9145f0 con 0x5651bdb4bbf0 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651bda386c0 con 0x5651bdb4c510 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb5b0f0 con 0x5651bdb584a0 2024-08-20T06:36:26.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb9c14740 10 client.0 _ll_register_callbacks cb 0x5651bda751e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.292+0000 7fdfb4db6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58740/0 (socket says 172.21.15.107:58740) 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 -- 172.21.15.107:0/4120311204 learned_addr learned my addr 172.21.15.107:0/4120311204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb55f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 -- 172.21.15.107:0/4120311204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 msgr2=0x5651bdb55f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb55f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 -- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 msgr2=0x5651bdb559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 -- 172.21.15.107:0/4120311204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bdb64520 con 0x5651bdb4bbf0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000afc0 tx=0x7fdfa00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46aed4f26582f4b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdf98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:26.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0019e60 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdfa0014a60 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac040070 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdfa0014c60 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb9c14740 10 client.15159 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.293+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651bdb66f30 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfac040070 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdfa0015040 con 0x5651bdb4bbf0 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 10 client.15159 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 10 client.15159 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 20 client.15159 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdb69550 0x5651bdb6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 4 client.15159 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:26.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651bdac45f0 con 0x5651bdb69550 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.294+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdb69550 0x5651bdb6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.295+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdb69550 0x5651bdb6a9c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21911023ef317a7d server_cookie=120dd6144c8cb399 in_seq=0 out_seq=0 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.295+0000 7fdf98ff9640 10 client.15159 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.295+0000 7fdf98ff9640 1 -- 172.21.15.107:0/4120311204 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651bdac45f0 con 0x5651bdb69550 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.295+0000 7fdf98ff9640 10 client.15159 handle_command_reply: tid=0 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.295+0000 7fdfb9c14740 1 client.15159 shutdown 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 msgr2=0x5651bdb55490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdfa000afc0 tx=0x7fdfa00047d0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 shutdown_connections 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb55f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdb69550 0x5651bdb6a9c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/4120311204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4bbf0 0x5651bdb55490 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdf98ff9640 0 client.15159 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 >> 172.21.15.107:0/4120311204 conn(0x5651bdb49a20 msgr2=0x5651bdb4f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 shutdown_connections 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 1 -- 172.21.15.107:0/4120311204 wait complete. 2024-08-20T06:36:26.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.296+0000 7fdfb9c14740 20 client.15159 trim_cache size 0 max 16384 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 Processor -- start 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 -- start start 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb4bbf0 0x5651bdb6a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb68d00 con 0x5651bdb584a0 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb693d0 con 0x5651bdb4c510 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb69aa0 con 0x5651bdb4bbf0 2024-08-20T06:36:26.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58752/0 (socket says 172.21.15.107:58752) 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 -- 172.21.15.107:0/3286867046 learned_addr learned my addr 172.21.15.107:0/3286867046 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb4bbf0 0x5651bdb6a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 -- 172.21.15.107:0/3286867046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb4bbf0 msgr2=0x5651bdb6a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb4bbf0 0x5651bdb6a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 -- 172.21.15.107:0/3286867046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 msgr2=0x5651bdb4fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb4fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 -- 172.21.15.107:0/3286867046 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651bdb49850 con 0x5651bdb584a0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0014760 tx=0x7fdfa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a654783d4e6f45 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.314+0000 7fdf9affd640 1 -- 172.21.15.107:0/3286867046 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0019e60 con 0x5651bdb584a0 2024-08-20T06:36:26.321 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdf9affd640 1 -- 172.21.15.107:0/3286867046 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfa001f070 con 0x5651bdb584a0 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdf9affd640 1 -- 172.21.15.107:0/3286867046 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0023af0 con 0x5651bdb584a0 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3286867046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 msgr2=0x5651bdb683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0014760 tx=0x7fdfa00046c0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3286867046 shutdown_connections 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb584a0 0x5651bdb683f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4c510 0x5651bdb4fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/3286867046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb4bbf0 0x5651bdb6a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3286867046 >> 172.21.15.107:0/3286867046 conn(0x5651bdb49a20 msgr2=0x5651bdb50700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3286867046 shutdown_connections 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- 172.21.15.107:0/3286867046 wait complete. 2024-08-20T06:36:26.322 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.315+0000 7fdfb9c14740 1 -- start start 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 Processor -- start 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb72e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb51f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb6a840 con 0x5651bdb4c510 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb6a170 con 0x5651bdb4bbf0 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5651bdb6af10 con 0x5651bdb584a0 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb9c14740 10 client.0 _ll_register_callbacks cb 0x7fdfa000a1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58756/0 (socket says 172.21.15.107:58756) 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 -- 172.21.15.107:0/2658162204 learned_addr learned my addr 172.21.15.107:0/2658162204 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:26.323 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb72e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 -- 172.21.15.107:0/2658162204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 msgr2=0x5651bdb51f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb51f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 -- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 msgr2=0x5651bdb72e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb72e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb55b7640 1 -- 172.21.15.107:0/2658162204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651bdb4a4a0 con 0x5651bdb4c510 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.316+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb55b7640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005e70 tx=0x7fdfac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6748393550285a81 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac035040 con 0x5651bdb4c510 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdfac004040 con 0x5651bdb4c510 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfa001f070 con 0x5651bdb4c510 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fdfac004260 con 0x5651bdb4c510 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 10 client.15165 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:26.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651bdb72500 con 0x5651bdb4c510 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfa001f070 con 0x5651bdb4c510 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fdfac05a470 con 0x5651bdb4c510 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 10 client.15165 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 10 client.15165 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 20 client.15165 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdbd7dc0 0x5651bdbd6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 4 client.15165 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.317+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "status"}) -- 0x5651bdac0090 con 0x5651bdbd7dc0 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdbd7dc0 0x5651bdbd6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdf98ff9640 10 client.15165 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb4db6640 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdbd7dc0 0x5651bdbd6ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ade04fe20bd7358 server_cookie=d037dc7f45fe616d in_seq=0 out_seq=0 2024-08-20T06:36:26.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdf98ff9640 1 -- 172.21.15.107:0/2658162204 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5651bdac0090 con 0x5651bdbd7dc0 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdf98ff9640 10 client.15165 handle_command_reply: tid=0 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 client.15165 shutdown 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 msgr2=0x5651bdb733f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fdfac005e70 tx=0x7fdfac02a410 comp rx=0 tx=0).stop 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 shutdown_connections 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5651bdb584a0 0x5651bdb51f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5651bdb4c510 0x5651bdb733f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5651bdbd7dc0 0x5651bdbd6ad0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 --2- 172.21.15.107:0/2658162204 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5651bdb4bbf0 0x5651bdb72e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdf98ff9640 0 client.15165 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 >> 172.21.15.107:0/2658162204 conn(0x5651bdb49a20 msgr2=0x5651bdb53ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.318+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 shutdown_connections 2024-08-20T06:36:26.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb9c14740 1 -- 172.21.15.107:0/2658162204 wait complete. 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb9c14740 20 client.15165 trim_cache size 0 max 16384 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24548, 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:26.327 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 77.355350174999998, 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 107.653646722 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:26.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 msgr2=0x7fdf9c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 0x7fdf9c0b6120 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800caa0 tx=0x7fdfa800a890 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 msgr2=0x7fdfb019e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfa800cad0 tx=0x7fdfa800cf90 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 shutdown_connections 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfb0107d60 0x7fdfb019e820 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfb01073e0 0x7fdfb0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdf9c0b3c60 0x7fdf9c0b6120 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 --2- 172.21.15.107:0/3622766278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfb01019d0 0x7fdfb0198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 >> 172.21.15.107:0/3622766278 conn(0x7fdfb00fd440 msgr2=0x7fdfb00ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 shutdown_connections 2024-08-20T06:36:26.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:26.319+0000 7fdfb781b640 1 -- 172.21.15.107:0/3622766278 wait complete. 2024-08-20T06:36:27.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-20T06:36:27.307 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:27.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:27 smithi107 ceph-mon[18495]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:27 smithi183 ceph-mon[22204]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:27 smithi193 ceph-mon[21944]: pgmap v315: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:27.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 -- 172.21.15.107:0/2842735504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c108e40 msgr2=0x7f4f4c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 --2- 172.21.15.107:0/2842735504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c108e40 0x7f4f4c10f640 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009ca0 tx=0x7f4f4402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 -- 172.21.15.107:0/2842735504 shutdown_connections 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 --2- 172.21.15.107:0/2842735504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c108e40 0x7f4f4c10f640 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 --2- 172.21.15.107:0/2842735504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c1084c0 0x7f4f4c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 --2- 172.21.15.107:0/2842735504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 -- 172.21.15.107:0/2842735504 >> 172.21.15.107:0/2842735504 conn(0x7f4f4c0fe500 msgr2=0x7f4f4c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 -- 172.21.15.107:0/2842735504 shutdown_connections 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.791+0000 7f4f51543640 1 -- 172.21.15.107:0/2842735504 wait complete. 2024-08-20T06:36:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 Processor -- start 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 -- start start 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c1084c0 0x7f4f4c199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c108e40 0x7f4f4c19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112a00 con 0x7f4f4c102ab0 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112880 con 0x7f4f4c108e40 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.792+0000 7f4f51543640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112b80 con 0x7f4f4c1084c0 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58776/0 (socket says 172.21.15.107:58776) 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 -- 172.21.15.107:0/3944295366 learned_addr learned my addr 172.21.15.107:0/3944295366 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c1084c0 0x7f4f4c199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c108e40 0x7f4f4c19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 -- 172.21.15.107:0/3944295366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c1084c0 msgr2=0x7f4f4c199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c1084c0 0x7f4f4c199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 -- 172.21.15.107:0/3944295366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c108e40 msgr2=0x7f4f4c19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c108e40 0x7f4f4c19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 -- 172.21.15.107:0/3944295366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c19efc0 con 0x7f4f4c102ab0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f4affd640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00b4a0 tx=0x7f4f3c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f33fff640 1 -- 172.21.15.107:0/3944295366 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c014070 con 0x7f4f4c102ab0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c19f2b0 con 0x7f4f4c102ab0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f33fff640 1 -- 172.21.15.107:0/3944295366 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3c010aa0 con 0x7f4f4c102ab0 2024-08-20T06:36:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f33fff640 1 -- 172.21.15.107:0/3944295366 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c017b70 con 0x7f4f4c102ab0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.793+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1a5970 con 0x7f4f4c102ab0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f33fff640 1 -- 172.21.15.107:0/3944295366 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f4f3c017070 con 0x7f4f4c102ab0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 Processor -- start 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 -- start start 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x559214161fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x55921416df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5592140d86a0 con 0x55921416e4a0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5592140c7ba0 con 0x559214162510 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f5393c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5592140cbbe0 con 0x559214161bf0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58792/0 (socket says 172.21.15.107:58792) 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/1964379279 learned_addr learned my addr 172.21.15.107:0/1964379279 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.796+0000 7f4f33fff640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 0x7f4f340b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4affd640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x55921416df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x559214161fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/1964379279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 msgr2=0x559214161fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x559214161fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/1964379279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 msgr2=0x55921416df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x55921416df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/1964379279 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559213e27540 con 0x55921416e4a0 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x559214161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:27.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4affd640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x55921416df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 0x7f4f340b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f33fff640 1 -- 172.21.15.107:0/3944295366 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4f3c0e3070 con 0x7f4f4c102ab0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000ac20 tx=0x7f4f4000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820ad120a02665d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f317fa640 1 -- 172.21.15.107:0/1964379279 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4001d070 con 0x55921416e4a0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f317fa640 1 -- 172.21.15.107:0/1964379279 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f40018ad0 con 0x55921416e4a0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.797+0000 7f4f317fa640 1 -- 172.21.15.107:0/1964379279 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40010960 con 0x55921416e4a0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- 172.21.15.107:0/1964379279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 msgr2=0x559214170890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000ac20 tx=0x7f4f4000f790 comp rx=0 tx=0).stop 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 0x7f4f340b6120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009430 tx=0x7f4f44031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- 172.21.15.107:0/1964379279 shutdown_connections 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55921416e4a0 0x559214170890 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x55921416df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1964379279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x559214161fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- 172.21.15.107:0/1964379279 >> 172.21.15.107:0/1964379279 conn(0x55921415fa20 msgr2=0x559214160bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- 172.21.15.107:0/1964379279 shutdown_connections 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- 172.21.15.107:0/1964379279 wait complete. 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 -- start start 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.798+0000 7f4f5393c740 1 Processor -- start 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x55921416b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55921416e4a0 0x55921416bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559213f2a5f0 con 0x559214162510 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55921404e6c0 con 0x55921416e4a0 2024-08-20T06:36:27.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559214171150 con 0x559214161bf0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 10 client.0 _ll_register_callbacks cb 0x55921408b1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58808/0 (socket says 172.21.15.107:58808) 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/3244485788 learned_addr learned my addr 172.21.15.107:0/3244485788 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 msgr2=0x55921416b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x55921416b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55921416e4a0 0x55921416bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/3244485788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55921416e4a0 msgr2=0x55921416bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55921416e4a0 0x55921416bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 -- 172.21.15.107:0/3244485788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55921417a580 con 0x559214162510 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000a980 tx=0x7f4f400186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dd60a35273a8de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:27.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40026020 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4f40005450 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f40018ad0 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4f4001d070 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 10 client.15177 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.799+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55921417ce80 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f40018ad0 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4f400046c0 con 0x559214162510 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 10 client.15177 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 10 client.15177 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 20 client.15177 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141805c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 4 client.15177 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:27.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592140da5f0 con 0x55921417f150 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.800+0000 7f4f4affd640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141805c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.801+0000 7f4f2eff5640 10 client.15177 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.801+0000 7f4f4affd640 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141805c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb4ee0d981e01f49 server_cookie=bea15113a7217c83 in_seq=0 out_seq=0 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.801+0000 7f4f2eff5640 1 -- 172.21.15.107:0/3244485788 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592140da5f0 con 0x55921417f150 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.801+0000 7f4f2eff5640 10 client.15177 handle_command_reply: tid=0 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.801+0000 7f4f5393c740 1 client.15177 shutdown 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 msgr2=0x55921416b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4f4000a980 tx=0x7f4f400186f0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 shutdown_connections 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141805c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55921416e4a0 0x55921416bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f2eff5640 0 client.15177 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214162510 0x55921416b9e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 --2- 172.21.15.107:0/3244485788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559214161bf0 0x55921416b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 >> 172.21.15.107:0/3244485788 conn(0x55921415fa20 msgr2=0x5592141655c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 shutdown_connections 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 1 -- 172.21.15.107:0/3244485788 wait complete. 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.802+0000 7f4f5393c740 20 client.15177 trim_cache size 0 max 16384 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 Processor -- start 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 -- start start 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214165db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214179af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55921417e900 con 0x559214161bf0 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55921417efd0 con 0x559214162510 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f5393c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55921417f6a0 con 0x55921416e4a0 2024-08-20T06:36:27.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58818/0 (socket says 172.21.15.107:58818) 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 -- 172.21.15.107:0/2430269466 learned_addr learned my addr 172.21.15.107:0/2430269466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 -- 172.21.15.107:0/2430269466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 msgr2=0x559214179af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214179af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 -- 172.21.15.107:0/2430269466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 msgr2=0x559214165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 -- 172.21.15.107:0/2430269466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55921416a400 con 0x559214161bf0 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f4affd640 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0e0b20 tx=0x7f4f3c0776d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34fe79a489cf10b5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.820+0000 7f4f30ff9640 1 -- 172.21.15.107:0/2430269466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01e040 con 0x559214161bf0 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f30ff9640 1 -- 172.21.15.107:0/2430269466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3c0ae530 con 0x559214161bf0 2024-08-20T06:36:27.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f30ff9640 1 -- 172.21.15.107:0/2430269466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c010c50 con 0x559214161bf0 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 -- 172.21.15.107:0/2430269466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 msgr2=0x559214169d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0e0b20 tx=0x7f4f3c0776d0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 -- 172.21.15.107:0/2430269466 shutdown_connections 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214179af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214165db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 --2- 172.21.15.107:0/2430269466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214169d20 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 -- 172.21.15.107:0/2430269466 >> 172.21.15.107:0/2430269466 conn(0x55921415fa20 msgr2=0x5592141793e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 -- 172.21.15.107:0/2430269466 shutdown_connections 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.821+0000 7f4f5393c740 1 -- 172.21.15.107:0/2430269466 wait complete. 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 -- start start 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 Processor -- start 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214167d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214168290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559214180440 con 0x559214161bf0 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55921417fd70 con 0x559214162510 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559214180b10 con 0x55921416e4a0 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f5393c740 10 client.0 _ll_register_callbacks cb 0x7f4f4001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58834/0 (socket says 172.21.15.107:58834) 2024-08-20T06:36:27.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 -- 172.21.15.107:0/1215524720 learned_addr learned my addr 172.21.15.107:0/1215524720 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214168290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 -- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 msgr2=0x559214168290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214168290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 -- 172.21.15.107:0/1215524720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 msgr2=0x559214167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214167d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 -- 172.21.15.107:0/1215524720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559214187990 con 0x559214161bf0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4b7fe640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f4affd640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0a9be0 tx=0x7f4f3c0b30d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e179635c4e713e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.822+0000 7f4f2eff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c01e050 con 0x559214161bf0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4f3c0a61c0 con 0x559214161bf0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f3c0ae530 con 0x559214161bf0 2024-08-20T06:36:27.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4f3c0e0c30 con 0x559214161bf0 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 10 client.15183 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559214188c90 con 0x559214161bf0 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4f3c0ae530 con 0x559214161bf0 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4f3c0a9190 con 0x559214161bf0 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 10 client.15183 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 10 client.15183 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 20 client.15183 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 4 client.15183 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.823+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "status"}) -- 0x5592140d6090 con 0x55921417f150 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f2eff5640 10 client.15183 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:27.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f4a7fc640 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141ec200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fabcad7687519b15 server_cookie=816c6634e1a4b8f9 in_seq=0 out_seq=0 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f2eff5640 1 -- 172.21.15.107:0/1215524720 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5592140d6090 con 0x55921417f150 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f2eff5640 10 client.15183 handle_command_reply: tid=0 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 client.15183 shutdown 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 msgr2=0x559214167810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c0a9be0 tx=0x7f4f3c0b30d0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 shutdown_connections 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55921417f150 0x5592141ec200 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55921416e4a0 0x559214168290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559214162510 0x559214167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 --2- 172.21.15.107:0/1215524720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559214161bf0 0x559214167810 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 >> 172.21.15.107:0/1215524720 conn(0x55921415fa20 msgr2=0x55921416cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f2eff5640 0 client.15183 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:27.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 shutdown_connections 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 1 -- 172.21.15.107:0/1215524720 wait complete. 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.824+0000 7f4f5393c740 20 client.15183 trim_cache size 0 max 16384 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24562, 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:27.843 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 79.869675201999996, 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 106.65184913 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:27.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 msgr2=0x7f4f340b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 0x7f4f340b6120 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009430 tx=0x7f4f44031000 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 msgr2=0x7f4f4c198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00b4a0 tx=0x7f4f3c00b960 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 shutdown_connections 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4f4c108e40 0x7f4f4c19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4f340b3c60 0x7f4f340b6120 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f4c1084c0 0x7f4f4c199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 --2- 172.21.15.107:0/3944295366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4f4c102ab0 0x7f4f4c198f10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 >> 172.21.15.107:0/3944295366 conn(0x7f4f4c0fe500 msgr2=0x7f4f4c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 shutdown_connections 2024-08-20T06:36:27.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:27.825+0000 7f4f51543640 1 -- 172.21.15.107:0/3944295366 wait complete. 2024-08-20T06:36:28.570 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T06:36:28.815 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 -- 172.21.15.107:0/4037572507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec1019a0 msgr2=0x7f40ec101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 --2- 172.21.15.107:0/4037572507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec1019a0 0x7f40ec101d80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f40e4009900 tx=0x7f40e402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 -- 172.21.15.107:0/4037572507 shutdown_connections 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 --2- 172.21.15.107:0/4037572507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec107d30 0x7f40ec10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 --2- 172.21.15.107:0/4037572507 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1073b0 0x7f40ec1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 --2- 172.21.15.107:0/4037572507 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec1019a0 0x7f40ec101d80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 -- 172.21.15.107:0/4037572507 >> 172.21.15.107:0/4037572507 conn(0x7f40ec0fd3f0 msgr2=0x7f40ec0ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 -- 172.21.15.107:0/4037572507 shutdown_connections 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.293+0000 7f40f166b640 1 -- 172.21.15.107:0/4037572507 wait complete. 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 Processor -- start 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 -- start start 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1019a0 0x7f40ec0fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec1073b0 0x7f40ec0fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec1118a0 con 0x7f40ec107d30 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec111720 con 0x7f40ec1019a0 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40f166b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f40ec111a20 con 0x7f40ec1073b0 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.294+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58848/0 (socket says 172.21.15.107:58848) 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1655123823 learned_addr learned my addr 172.21.15.107:0/1655123823 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1019a0 0x7f40ec0fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec1073b0 0x7f40ec0fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1655123823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec1073b0 msgr2=0x7f40ec0fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec1073b0 0x7f40ec0fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1655123823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1019a0 msgr2=0x7f40ec0fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1019a0 0x7f40ec0fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1655123823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40ec1a7350 con 0x7f40ec107d30 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b970 tx=0x7f40dc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40e3fff640 1 -- 172.21.15.107:0/1655123823 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc019070 con 0x7f40ec107d30 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40ec1a7640 con 0x7f40ec107d30 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40e3fff640 1 -- 172.21.15.107:0/1655123823 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40dc014440 con 0x7f40ec107d30 2024-08-20T06:36:29.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40e3fff640 1 -- 172.21.15.107:0/1655123823 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0075d0 con 0x7f40ec107d30 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.295+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40ec1a7c00 con 0x7f40ec107d30 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 Processor -- start 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40e3fff640 1 -- 172.21.15.107:0/1655123823 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f40dc01c070 con 0x7f40ec107d30 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 -- start start 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0785fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b0794890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0b06fc6a0 con 0x55a0b0786510 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0b06ebba0 con 0x55a0b0785bf0 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40f3a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0b06efbe0 con 0x55a0b07924a0 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.298+0000 7f40eaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58854/0 (socket says 172.21.15.107:58854) 2024-08-20T06:36:29.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 -- 172.21.15.107:0/836214550 learned_addr learned my addr 172.21.15.107:0/836214550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0785fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b0794890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 -- 172.21.15.107:0/836214550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 msgr2=0x55a0b0794890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b0794890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 -- 172.21.15.107:0/836214550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 msgr2=0x55a0b0785fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0785fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 -- 172.21.15.107:0/836214550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b044b540 con 0x55a0b0786510 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40e3fff640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 0x7f40d80b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40eaffd640 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f40e402a930 tx=0x7f40e4031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e42ee035f25ee63 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 0x7f40d80b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40e17fa640 1 -- 172.21.15.107:0/836214550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4035070 con 0x55a0b0786510 2024-08-20T06:36:29.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40e3fff640 1 -- 172.21.15.107:0/1655123823 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40dc0e3220 con 0x7f40ec107d30 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40e17fa640 1 -- 172.21.15.107:0/836214550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40e4040070 con 0x55a0b0786510 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.299+0000 7f40e17fa640 1 -- 172.21.15.107:0/836214550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003a10 con 0x55a0b0786510 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 -- 172.21.15.107:0/836214550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 msgr2=0x55a0b0791f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 0x7f40d80b6120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55a0b0785ba0 tx=0x7f40d4004730 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f40e402a930 tx=0x7f40e4031dc0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 -- 172.21.15.107:0/836214550 shutdown_connections 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b0794890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0791f60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 --2- 172.21.15.107:0/836214550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0785fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 -- 172.21.15.107:0/836214550 >> 172.21.15.107:0/836214550 conn(0x55a0b0783a20 msgr2=0x55a0b0784bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 -- 172.21.15.107:0/836214550 shutdown_connections 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.300+0000 7f40f3a64740 1 -- 172.21.15.107:0/836214550 wait complete. 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 -- start start 2024-08-20T06:36:29.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 Processor -- start 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b078f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b07924a0 0x55a0b078fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0b054e5f0 con 0x55a0b0786510 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0b06726c0 con 0x55a0b07924a0 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07950f0 con 0x55a0b0785bf0 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40f3a64740 10 client.0 _ll_register_callbacks cb 0x55a0b06af1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58866/0 (socket says 172.21.15.107:58866) 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 -- 172.21.15.107:0/1549021623 learned_addr learned my addr 172.21.15.107:0/1549021623 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b07924a0 0x55a0b078fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b078f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 -- 172.21.15.107:0/1549021623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 msgr2=0x55a0b078f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b078f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 -- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b07924a0 msgr2=0x55a0b078fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b07924a0 0x55a0b078fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.301+0000 7f40ea7fc640 1 -- 172.21.15.107:0/1549021623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b079e4a0 con 0x55a0b0786510 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f40d4014fb0 tx=0x7f40d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373925be34d62083 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d400fd90 con 0x55a0b0786510 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40d4004020 con 0x55a0b0786510 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e4040070 con 0x55a0b0786510 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f40d4004220 con 0x55a0b0786510 2024-08-20T06:36:29.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 10 client.15195 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a0b07a0eb0 con 0x55a0b0786510 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40e4040070 con 0x55a0b0786510 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f40d4011440 con 0x55a0b0786510 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 10 client.15195 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 10 client.15195 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 20 client.15195 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b07a3550 0x55a0b07a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 4 client.15195 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.302+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0b06fe5f0 con 0x55a0b07a3550 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.303+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b07a3550 0x55a0b07a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.303+0000 7f40d27fc640 10 client.15195 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:29.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.303+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b07a3550 0x55a0b07a49c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223e7843dd99f46f server_cookie=892573af4f0855ce in_seq=0 out_seq=0 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40d27fc640 1 -- 172.21.15.107:0/1549021623 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0b06fe5f0 con 0x55a0b07a3550 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40d27fc640 10 client.15195 handle_command_reply: tid=0 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 client.15195 shutdown 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 msgr2=0x55a0b078f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f40d4014fb0 tx=0x7f40d40046c0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 shutdown_connections 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b07924a0 0x55a0b078fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b078f940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b07a3550 0x55a0b07a49c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1549021623 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b078f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40d27fc640 0 client.15195 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.304+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 >> 172.21.15.107:0/1549021623 conn(0x55a0b0783a20 msgr2=0x55a0b07895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.305+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 shutdown_connections 2024-08-20T06:36:29.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.305+0000 7f40f3a64740 1 -- 172.21.15.107:0/1549021623 wait complete. 2024-08-20T06:36:29.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.305+0000 7f40f3a64740 20 client.15195 trim_cache size 0 max 16384 2024-08-20T06:36:29.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 Processor -- start 2024-08-20T06:36:29.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 -- start start 2024-08-20T06:36:29.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0789db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b07acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a2d00 con 0x55a0b0786510 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a33d0 con 0x55a0b0785bf0 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40f3a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a3aa0 con 0x55a0b07924a0 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40ea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40ea7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58870/0 (socket says 172.21.15.107:58870) 2024-08-20T06:36:29.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40ea7fc640 1 -- 172.21.15.107:0/2673082506 learned_addr learned my addr 172.21.15.107:0/2673082506 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.323+0000 7f40eaffd640 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0789db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b07acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 -- 172.21.15.107:0/2673082506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 msgr2=0x55a0b07acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b07acca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 -- 172.21.15.107:0/2673082506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 msgr2=0x55a0b0789db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0789db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 -- 172.21.15.107:0/2673082506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0b078d900 con 0x55a0b0786510 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40ea7fc640 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f40d4014440 tx=0x7f40d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1e56a53121e10 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40e0ff9640 1 -- 172.21.15.107:0/2673082506 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d400cb50 con 0x55a0b0786510 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40e0ff9640 1 -- 172.21.15.107:0/2673082506 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40d40043c0 con 0x55a0b0786510 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.324+0000 7f40e0ff9640 1 -- 172.21.15.107:0/2673082506 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4033670 con 0x55a0b0786510 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- 172.21.15.107:0/2673082506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 msgr2=0x55a0b0785180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f40d4014440 tx=0x7f40d400c8d0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- 172.21.15.107:0/2673082506 shutdown_connections 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b07924a0 0x55a0b07acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b0786510 0x55a0b0785180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- 172.21.15.107:0/2673082506 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0785bf0 0x55a0b0789db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- 172.21.15.107:0/2673082506 >> 172.21.15.107:0/2673082506 conn(0x55a0b0783a20 msgr2=0x55a0b0784a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- 172.21.15.107:0/2673082506 shutdown_connections 2024-08-20T06:36:29.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- 172.21.15.107:0/2673082506 wait complete. 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- start start 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 Processor -- start 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b07a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0786510 0x55a0b07a8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a4840 con 0x55a0b07924a0 2024-08-20T06:36:29.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a4170 con 0x55a0b0786510 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.325+0000 7f40f3a64740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55a0b07a4f10 con 0x55a0b0785bf0 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40f3a64740 10 client.0 _ll_register_callbacks cb 0x7f40d4014db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40ea7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0786510 0x55a0b07a8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58880/0 (socket says 172.21.15.107:58880) 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1382944299 learned_addr learned my addr 172.21.15.107:0/1382944299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:29.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b07a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1382944299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 msgr2=0x55a0b07a83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b07a83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0786510 msgr2=0x55a0b07a8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0786510 0x55a0b07a8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 -- 172.21.15.107:0/1382944299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0b080c340 con 0x55a0b07924a0 2024-08-20T06:36:29.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b07a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40eb7fe640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0a6250 tx=0x7f40dc0e1050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818941fbae73159d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc0b2d40 con 0x55a0b07924a0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40dc0b2ee0 con 0x55a0b07924a0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d40043c0 con 0x55a0b07924a0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f40dc00cb40 con 0x55a0b07924a0 2024-08-20T06:36:29.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40f3a64740 10 client.15201 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.326+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a0b080d6a0 con 0x55a0b07924a0 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40d40043c0 con 0x55a0b07924a0 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f40dc0a5bd0 con 0x55a0b07924a0 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 10 client.15201 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 10 client.15201 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 20 client.15201 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:29.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b0812240 0x55a0b0810f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 4 client.15201 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "status"}) -- 0x55a0b06fa090 con 0x55a0b0812240 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b0812240 0x55a0b0810f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40eaffd640 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b0812240 0x55a0b0810f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d50d0728c07177e1 server_cookie=36b563d222840c1e in_seq=0 out_seq=0 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.327+0000 7f40d27fc640 10 client.15201 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40d27fc640 1 -- 172.21.15.107:0/1382944299 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a0b06fa090 con 0x55a0b0812240 2024-08-20T06:36:29.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40d27fc640 10 client.15201 handle_command_reply: tid=0 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 client.15201 shutdown 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 msgr2=0x55a0b080bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f40dc0a6250 tx=0x7f40dc0e1050 comp rx=0 tx=0).stop 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 shutdown_connections 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55a0b07924a0 0x55a0b080bc50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55a0b0786510 0x55a0b07a8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55a0b0812240 0x55a0b0810f50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 --2- 172.21.15.107:0/1382944299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0b0785bf0 0x55a0b07a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 >> 172.21.15.107:0/1382944299 conn(0x55a0b0783a20 msgr2=0x55a0b0790a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40d27fc640 0 client.15201 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 shutdown_connections 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 1 -- 172.21.15.107:0/1382944299 wait complete. 2024-08-20T06:36:29.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f3a64740 20 client.15201 trim_cache size 0 max 16384 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24572, 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 88.412699678999999, 2024-08-20T06:36:29.344 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:29.345 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 99.025787381000001 2024-08-20T06:36:29.345 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 msgr2=0x7f40d80b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 0x7f40d80b6120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x55a0b0785ba0 tx=0x7f40d4004730 comp rx=0 tx=0).stop 2024-08-20T06:36:29.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 msgr2=0x7f40ec0fe640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.328+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b970 tx=0x7f40dc00be30 comp rx=0 tx=0).stop 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 shutdown_connections 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40ec107d30 0x7f40ec0fe640 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40d80b3c60 0x7f40d80b6120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40ec1073b0 0x7f40ec0fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 --2- 172.21.15.107:0/1655123823 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40ec1019a0 0x7f40ec0fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 >> 172.21.15.107:0/1655123823 conn(0x7f40ec0fd3f0 msgr2=0x7f40ec06b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:29.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 shutdown_connections 2024-08-20T06:36:29.348 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:29.329+0000 7f40f166b640 1 -- 172.21.15.107:0/1655123823 wait complete. 2024-08-20T06:36:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:29 smithi183 ceph-mon[22204]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:36:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:29 smithi193 ceph-mon[21944]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:36:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:29 smithi107 ceph-mon[18495]: pgmap v316: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:36:30.047 INFO:tasks.cephadm:Setup complete, yielding 2024-08-20T06:36:30.047 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T06:36:30.065 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-08-20T06:36:30.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-20T06:36:30.311 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 -- 172.21.15.107:0/839991224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac104260 msgr2=0x7f8aac10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 --2- 172.21.15.107:0/839991224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac104260 0x7f8aac10abe0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009cd0 tx=0x7f8aa402a8e0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 -- 172.21.15.107:0/839991224 shutdown_connections 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 --2- 172.21.15.107:0/839991224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac104260 0x7f8aac10abe0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 --2- 172.21.15.107:0/839991224 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac1038a0 0x7f8aac103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 --2- 172.21.15.107:0/839991224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac102740 0x7f8aac102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 -- 172.21.15.107:0/839991224 >> 172.21.15.107:0/839991224 conn(0x7f8aac0fdff0 msgr2=0x7f8aac100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 -- 172.21.15.107:0/839991224 shutdown_connections 2024-08-20T06:36:30.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.817+0000 7f8ab2971640 1 -- 172.21.15.107:0/839991224 wait complete. 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 Processor -- start 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 -- start start 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 0x7f8aac1989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac104260 0x7f8aac199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10e250 con 0x7f8aac1038a0 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10e0d0 con 0x7f8aac102740 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab2971640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8aac10e3d0 con 0x7f8aac104260 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58898/0 (socket says 172.21.15.107:58898) 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 -- 172.21.15.107:0/4118633970 learned_addr learned my addr 172.21.15.107:0/4118633970 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8ab0f0e640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac104260 0x7f8aac199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:30.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aabfff640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 0x7f8aac1989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 -- 172.21.15.107:0/4118633970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac104260 msgr2=0x7f8aac199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac104260 0x7f8aac199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 -- 172.21.15.107:0/4118633970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 msgr2=0x7f8aac1989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 0x7f8aac1989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aab7fe640 1 -- 172.21.15.107:0/4118633970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aac19a500 con 0x7f8aac1038a0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.818+0000 7f8aabfff640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 0x7f8aac1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8aab7fe640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c890 tx=0x7f8aa000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004220 con 0x7f8aac1038a0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8aa0007e40 con 0x7f8aac1038a0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aac19a7f0 con 0x7f8aac1038a0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0004a00 con 0x7f8aac1038a0 2024-08-20T06:36:30.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.819+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aac1a0ed0 con 0x7f8aac1038a0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.820+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aac102b40 con 0x7f8aac1038a0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.821+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f8aa00079b0 con 0x7f8aac1038a0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.822+0000 7f8aa97fa640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 0x7f8a940b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.822+0000 7f8aabfff640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 0x7f8a940b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.822+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8aa00e3f60 con 0x7f8aac1038a0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.822+0000 7f8aabfff640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 0x7f8a940b6120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0058c0 tx=0x7f8a9c005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:30.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:30.825+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8aa0014030 con 0x7f8aac1038a0 2024-08-20T06:36:31.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.016+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8aac0632c0 con 0x7f8a940b3c60 2024-08-20T06:36:31.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.017+0000 7f8aa97fa640 1 -- 172.21.15.107:0/4118633970 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8aac0632c0 con 0x7f8a940b3c60 2024-08-20T06:36:31.018 INFO:teuthology.orchestra.run.smithi107.stdout:Backend: cephadm 2024-08-20T06:36:31.018 INFO:teuthology.orchestra.run.smithi107.stdout:Available: Yes 2024-08-20T06:36:31.018 INFO:teuthology.orchestra.run.smithi107.stdout:Paused: No 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 msgr2=0x7f8a940b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 0x7f8a940b6120 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0058c0 tx=0x7f8a9c005d30 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 msgr2=0x7f8aac199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c890 tx=0x7f8aa000cd50 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 shutdown_connections 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a940b3c60 0x7f8a940b6120 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aac104260 0x7f8aac199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8aac1038a0 0x7f8aac199020 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 --2- 172.21.15.107:0/4118633970 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8aac102740 0x7f8aac1989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 >> 172.21.15.107:0/4118633970 conn(0x7f8aac0fdff0 msgr2=0x7f8aac106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:31.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 shutdown_connections 2024-08-20T06:36:31.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:31.020+0000 7f8ab2971640 1 -- 172.21.15.107:0/4118633970 wait complete. 2024-08-20T06:36:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:31 smithi107 ceph-mon[18495]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:31 smithi183 ceph-mon[22204]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:31 smithi193 ceph-mon[21944]: pgmap v317: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:31.669 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-20T06:36:31.914 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:32.329 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:32 smithi107 ceph-mon[18495]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- 172.21.15.107:0/1999346444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c103130 msgr2=0x7f2c7c109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- 172.21.15.107:0/1999346444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c103130 0x7f2c7c109ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009cd0 tx=0x7f2c7402a8e0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- 172.21.15.107:0/1999346444 shutdown_connections 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- 172.21.15.107:0/1999346444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c103130 0x7f2c7c109ab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- 172.21.15.107:0/1999346444 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c102770 0x7f2c7c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- 172.21.15.107:0/1999346444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c101610 0x7f2c7c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- 172.21.15.107:0/1999346444 >> 172.21.15.107:0/1999346444 conn(0x7f2c7c0fcee0 msgr2=0x7f2c7c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- 172.21.15.107:0/1999346444 shutdown_connections 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- 172.21.15.107:0/1999346444 wait complete. 2024-08-20T06:36:32.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 Processor -- start 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- start start 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101610 0x7f2c7c06b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 0x7f2c7c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d1a0 con 0x7f2c7c102770 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d020 con 0x7f2c7c101610 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c82ade640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2c7c10d320 con 0x7f2c7c103130 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.409+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58928/0 (socket says 172.21.15.107:58928) 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 -- 172.21.15.107:0/901168850 learned_addr learned my addr 172.21.15.107:0/901168850 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c8087a640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101610 0x7f2c7c06b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:32.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c8107b640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 0x7f2c7c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 -- 172.21.15.107:0/901168850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 msgr2=0x7f2c7c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 0x7f2c7c19fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 -- 172.21.15.107:0/901168850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101610 msgr2=0x7f2c7c06b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101610 0x7f2c7c06b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 -- 172.21.15.107:0/901168850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7c1a0620 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c8107b640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 0x7f2c7c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c7bfff640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b4a0 tx=0x7f2c6800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68014070 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7c1a0910 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2c68010aa0 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c68017b70 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.410+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7c1a0e50 con 0x7f2c7c102770 2024-08-20T06:36:32.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.411+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c7c108990 con 0x7f2c7c102770 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.412+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7f2c68017070 con 0x7f2c7c102770 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.414+0000 7f2c79ffb640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 0x7f2c6c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.414+0000 7f2c8087a640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 0x7f2c6c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.414+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2c680e33d0 con 0x7f2c7c102770 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.414+0000 7f2c8087a640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 0x7f2c6c0b6120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c700058c0 tx=0x7f2c7000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:32.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.418+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2c680aa9d0 con 0x7f2c7c102770 2024-08-20T06:36:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:32 smithi183 ceph-mon[22204]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:32.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:32 smithi193 ceph-mon[21944]: from='client.15207 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:32.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.602+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2c7c0632c0 con 0x7f2c6c0b3c60 2024-08-20T06:36:32.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.614+0000 7f2c79ffb640 1 -- 172.21.15.107:0/901168850 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2c7c0632c0 con 0x7f2c6c0b3c60 2024-08-20T06:36:32.615 INFO:teuthology.orchestra.run.smithi107.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-20T06:36:32.615 INFO:teuthology.orchestra.run.smithi107.stdout:mds.a smithi107 running (105s) 26s ago 104s 14.5M - 19.0.0-5872-gef427541 9f5c88d94bb6 0e539f4dd9a5 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.b smithi183 running (102s) 35s ago 102s 26.7M - 19.0.0-5872-gef427541 9f5c88d94bb6 13b4c6f738a4 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.c smithi193 running (116s) 35s ago 116s 20.6M - 19.0.0-5872-gef427541 9f5c88d94bb6 58f45131e0ee 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.d smithi107 running (2m) 26s ago 2m 14.6M - 19.0.0-5872-gef427541 9f5c88d94bb6 2815632f1556 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.e smithi183 running (2m) 35s ago 2m 19.3M - 19.0.0-5872-gef427541 9f5c88d94bb6 627770a2698c 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.f smithi193 running (118s) 35s ago 118s 17.2M - 19.0.0-5872-gef427541 9f5c88d94bb6 efcd3257311e 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.g smithi107 running (108s) 26s ago 108s 19.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 927dcf36df9c 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.h smithi183 running (114s) 35s ago 113s 18.1M - 19.0.0-5872-gef427541 9f5c88d94bb6 6a963b9ed155 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.i smithi193 running (111s) 35s ago 111s 17.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 9279fe3300e0 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.j smithi107 running (2m) 26s ago 2m 13.9M - 19.0.0-5872-gef427541 9f5c88d94bb6 bee74c5069bd 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.k smithi183 running (2m) 35s ago 2m 14.6M - 19.0.0-5872-gef427541 9f5c88d94bb6 405c72118051 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mds.l smithi193 running (2m) 35s ago 2m 16.9M - 19.0.0-5872-gef427541 9f5c88d94bb6 f18520c57ef8 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.x smithi107 *:9283,8765 running (9m) 26s ago 9m 521M - 19.0.0-5872-gef427541 9f5c88d94bb6 b70a6c5efcff 2024-08-20T06:36:32.616 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.y smithi183 *:8443,8765 running (7m) 35s ago 7m 454M - 19.0.0-5872-gef427541 9f5c88d94bb6 c1775697a9ea 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:mgr.z smithi193 *:8443,8765 running (7m) 35s ago 7m 455M - 19.0.0-5872-gef427541 9f5c88d94bb6 b4d8ddeaf9e1 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:mon.a smithi107 running (9m) 26s ago 9m 51.4M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 b8c843f327a7 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:mon.b smithi183 running (7m) 35s ago 7m 40.9M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 a129e4ad13be 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:mon.c smithi193 running (7m) 35s ago 7m 40.5M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 c44fe035ac45 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 smithi107 running (6m) 26s ago 6m 55.7M 4271M 19.0.0-5872-gef427541 9f5c88d94bb6 36853bc885a1 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 smithi183 running (6m) 35s ago 6m 49.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 df9449258bae 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 smithi193 running (6m) 35s ago 6m 57.8M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 e610b98d865a 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 smithi107 running (5m) 26s ago 5m 46.4M 4271M 19.0.0-5872-gef427541 9f5c88d94bb6 18c8925da28e 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 smithi183 running (5m) 35s ago 5m 53.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 247fe6ef5329 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 smithi193 running (4m) 35s ago 4m 48.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 221848fc596b 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 smithi107 running (4m) 26s ago 4m 45.1M 4271M 19.0.0-5872-gef427541 9f5c88d94bb6 192a52db8b56 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 smithi183 running (4m) 35s ago 4m 53.2M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 81301b6d383b 2024-08-20T06:36:32.617 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 smithi193 running (3m) 35s ago 3m 55.9M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 7faf500d55d0 2024-08-20T06:36:32.618 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 smithi107 running (3m) 26s ago 3m 52.0M 4271M 19.0.0-5872-gef427541 9f5c88d94bb6 e53995f27797 2024-08-20T06:36:32.618 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 smithi183 running (2m) 35s ago 2m 48.7M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 ca33e163b5d6 2024-08-20T06:36:32.618 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 smithi193 running (2m) 35s ago 2m 45.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 301c04c6cd02 2024-08-20T06:36:32.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.616+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 msgr2=0x7f2c6c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:32.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.616+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 0x7f2c6c0b6120 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2c700058c0 tx=0x7f2c7000a4c0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.616+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 msgr2=0x7f2c7c19d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.616+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800b4a0 tx=0x7f2c6800b960 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 shutdown_connections 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2c6c0b3c60 0x7f2c6c0b6120 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c7c103130 0x7f2c7c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2c7c102770 0x7f2c7c19d770 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 --2- 172.21.15.107:0/901168850 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2c7c101610 0x7f2c7c06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 >> 172.21.15.107:0/901168850 conn(0x7f2c7c0fcee0 msgr2=0x7f2c7c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 shutdown_connections 2024-08-20T06:36:32.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:32.617+0000 7f2c82ade640 1 -- 172.21.15.107:0/901168850 wait complete. 2024-08-20T06:36:33.288 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-20T06:36:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:33 smithi183 ceph-mon[22204]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:33.533 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:33 smithi193 ceph-mon[21944]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:33 smithi107 ceph-mon[18495]: pgmap v318: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:34.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 -- 172.21.15.107:0/2914978674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8601038a0 msgr2=0x7ff860103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 --2- 172.21.15.107:0/2914978674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8601038a0 0x7ff860103d20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff854009930 tx=0x7ff85402a420 comp rx=0 tx=0).stop 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 -- 172.21.15.107:0/2914978674 shutdown_connections 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 --2- 172.21.15.107:0/2914978674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860104260 0x7ff86010abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 --2- 172.21.15.107:0/2914978674 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8601038a0 0x7ff860103d20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 --2- 172.21.15.107:0/2914978674 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff860102740 0x7ff860102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.017+0000 7ff867322640 1 -- 172.21.15.107:0/2914978674 >> 172.21.15.107:0/2914978674 conn(0x7ff8600fdff0 msgr2=0x7ff860100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- 172.21.15.107:0/2914978674 shutdown_connections 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- 172.21.15.107:0/2914978674 wait complete. 2024-08-20T06:36:34.030 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 Processor -- start 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- start start 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 0x7ff860198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8601038a0 0x7ff8601990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010e2d0 con 0x7ff860104260 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010e150 con 0x7ff8601038a0 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff867322640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff86010e450 con 0x7ff860102740 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff8658bf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff8658bf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58932/0 (socket says 172.21.15.107:58932) 2024-08-20T06:36:34.031 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.018+0000 7ff8658bf640 1 -- 172.21.15.107:0/1916854130 learned_addr learned my addr 172.21.15.107:0/1916854130 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8650be640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 0x7ff860198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8648bd640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8601038a0 0x7ff8601990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 -- 172.21.15.107:0/1916854130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 msgr2=0x7ff860198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 0x7ff860198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 -- 172.21.15.107:0/1916854130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8601038a0 msgr2=0x7ff8601990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8601038a0 0x7ff8601990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 -- 172.21.15.107:0/1916854130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff86019a5c0 con 0x7ff860104260 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8650be640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 0x7ff860198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff8658bf640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff85000eaa0 tx=0x7ff85000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:34.032 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85000cdb0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff850005020 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500116d0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff86019a8b0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.019+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8601a0f90 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.020+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff860109ae0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.021+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 17) ==== 151111+0+0 (secure 0 0 0) 0x7ff8500051d0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.022+0000 7ff85e7fc640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 0x7ff8480b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.022+0000 7ff8650be640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 0x7ff8480b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.022+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff8500e3fd0 con 0x7ff860104260 2024-08-20T06:36:34.033 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.022+0000 7ff8650be640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 0x7ff8480b61a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff858009930 tx=0x7ff858008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:34.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.026+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff850016030 con 0x7ff860104260 2024-08-20T06:36:34.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.213+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7ff8500a7050 con 0x7ff860104260 2024-08-20T06:36:34.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.216+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7ff8600632c0 con 0x7ff8480b3ce0 2024-08-20T06:36:34.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.222+0000 7ff85e7fc640 1 -- 172.21.15.107:0/1916854130 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7ff8600632c0 con 0x7ff8480b3ce0 2024-08-20T06:36:34.223 INFO:teuthology.orchestra.run.smithi107.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-20T06:36:34.223 INFO:teuthology.orchestra.run.smithi107.stdout:mds.all 12/12 37s ago 2m smithi107=a;smithi107=d;smithi107=g;smithi107=j;smithi183=b;smithi183=e;smithi183=h;smithi183=k;smithi193=c;smithi193=f;smithi193=i;smithi193=l;count:12 2024-08-20T06:36:34.223 INFO:teuthology.orchestra.run.smithi107.stdout:mgr 3/3 37s ago 7m smithi107=x;smithi183=y;smithi193=z;count:3 2024-08-20T06:36:34.223 INFO:teuthology.orchestra.run.smithi107.stdout:mon 3/3 37s ago 8m smithi107:172.21.15.107=a;smithi183:172.21.15.183=b;smithi193:172.21.15.193=c;count:3 2024-08-20T06:36:34.223 INFO:teuthology.orchestra.run.smithi107.stdout:osd 12 37s ago - 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 msgr2=0x7ff8480b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 0x7ff8480b61a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff858009930 tx=0x7ff858008040 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 msgr2=0x7ff860199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff85000eaa0 tx=0x7ff85000ef60 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 shutdown_connections 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff860104260 0x7ff860199d20 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8601038a0 0x7ff8601990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8480b3ce0 0x7ff8480b61a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 --2- 172.21.15.107:0/1916854130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff860102740 0x7ff860198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 >> 172.21.15.107:0/1916854130 conn(0x7ff8600fdff0 msgr2=0x7ff860106cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:34.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 shutdown_connections 2024-08-20T06:36:34.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:34.224+0000 7ff867322640 1 -- 172.21.15.107:0/1916854130 wait complete. 2024-08-20T06:36:34.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:34 smithi183 ceph-mon[22204]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:34.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:34 smithi107 ceph-mon[18495]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:34.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:34 smithi193 ceph-mon[21944]: from='client.15213 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:34.844 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-20T06:36:35.088 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:35 smithi183 ceph-mon[22204]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:35 smithi183 ceph-mon[22204]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-20T06:36:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:35 smithi183 ceph-mon[22204]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:35 smithi107 ceph-mon[18495]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:35 smithi107 ceph-mon[18495]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-20T06:36:35.504 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:35 smithi107 ceph-mon[18495]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:35 smithi193 ceph-mon[21944]: pgmap v319: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:35 smithi193 ceph-mon[21944]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-08-20T06:36:35.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:35 smithi193 ceph-mon[21944]: from='client.15219 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:35.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 -- 172.21.15.107:0/1608747349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc1027b0 msgr2=0x7f11dc102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:35.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 --2- 172.21.15.107:0/1608747349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc1027b0 0x7f11dc102c30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009930 tx=0x7f11c402a420 comp rx=0 tx=0).stop 2024-08-20T06:36:35.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 -- 172.21.15.107:0/1608747349 shutdown_connections 2024-08-20T06:36:35.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 --2- 172.21.15.107:0/1608747349 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc103170 0x7f11dc109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 --2- 172.21.15.107:0/1608747349 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc1027b0 0x7f11dc102c30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 --2- 172.21.15.107:0/1608747349 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc101650 0x7f11dc101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 -- 172.21.15.107:0/1608747349 >> 172.21.15.107:0/1608747349 conn(0x7f11dc0fcf00 msgr2=0x7f11dc0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 -- 172.21.15.107:0/1608747349 shutdown_connections 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.584+0000 7f11e258e640 1 -- 172.21.15.107:0/1608747349 wait complete. 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 Processor -- start 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 -- start start 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc1027b0 0x7f11dc1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 0x7f11dc199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc10d1e0 con 0x7f11dc101650 2024-08-20T06:36:35.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc10d060 con 0x7f11dc1027b0 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc10d360 con 0x7f11dc103170 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34266/0 (socket says 172.21.15.107:34266) 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 -- 172.21.15.107:0/3564590978 learned_addr learned my addr 172.21.15.107:0/3564590978 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11d3fff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc1027b0 0x7f11dc1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e0b2b640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 0x7f11dc199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 -- 172.21.15.107:0/3564590978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 msgr2=0x7f11dc199df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 0x7f11dc199df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 -- 172.21.15.107:0/3564590978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc1027b0 msgr2=0x7f11dc1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc1027b0 0x7f11dc1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 -- 172.21.15.107:0/3564590978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc19a690 con 0x7f11dc101650 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e0b2b640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 0x7f11dc199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:35.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11dbfff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b4a0 tx=0x7f11cc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:35.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc014070 con 0x7f11dc101650 2024-08-20T06:36:35.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f11cc010aa0 con 0x7f11dc101650 2024-08-20T06:36:35.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc017b70 con 0x7f11dc101650 2024-08-20T06:36:35.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.585+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc19a980 con 0x7f11dc101650 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.586+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc1a1060 con 0x7f11dc101650 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.587+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11dc1a1230 con 0x7f11dc101650 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.588+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f11cc017070 con 0x7f11dc101650 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.588+0000 7f11d9ffb640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 0x7f11c80b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.588+0000 7f11d3fff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 0x7f11c80b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.589+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f11cc0e2fd0 con 0x7f11dc101650 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.589+0000 7f11d3fff640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 0x7f11c80b6110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4005720 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:35.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.592+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f11cc0aa620 con 0x7f11dc101650 2024-08-20T06:36:35.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.773+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f11dc1a1400 con 0x7f11c80b3c50 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.774+0000 7f11d9ffb640 1 -- 172.21.15.107:0/3564590978 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f11dc1a1400 con 0x7f11c80b3c50 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:HOST ADDR LABELS STATUS 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 172.21.15.107 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:smithi183 172.21.15.183 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:smithi193 172.21.15.193 2024-08-20T06:36:35.775 INFO:teuthology.orchestra.run.smithi107.stdout:3 hosts in cluster 2024-08-20T06:36:35.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 msgr2=0x7f11c80b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:35.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 0x7f11c80b6110 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4005720 comp rx=0 tx=0).stop 2024-08-20T06:36:35.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 msgr2=0x7f11dc198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00b4a0 tx=0x7f11cc00b960 comp rx=0 tx=0).stop 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 shutdown_connections 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f11c80b3c50 0x7f11c80b6110 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11dc103170 0x7f11dc199df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f11dc1027b0 0x7f11dc1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 --2- 172.21.15.107:0/3564590978 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f11dc101650 0x7f11dc198b60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 >> 172.21.15.107:0/3564590978 conn(0x7f11dc0fcf00 msgr2=0x7f11dc06b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 shutdown_connections 2024-08-20T06:36:35.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:35.776+0000 7f11e258e640 1 -- 172.21.15.107:0/3564590978 wait complete. 2024-08-20T06:36:36.413 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-20T06:36:36.662 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:37.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 -- 172.21.15.107:0/1731373545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 msgr2=0x7fb440102c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:37.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 --2- 172.21.15.107:0/1731373545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb440102c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb428009930 tx=0x7fb42802a420 comp rx=0 tx=0).stop 2024-08-20T06:36:37.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 -- 172.21.15.107:0/1731373545 shutdown_connections 2024-08-20T06:36:37.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 --2- 172.21.15.107:0/1731373545 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb440109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 --2- 172.21.15.107:0/1731373545 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb440102c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 --2- 172.21.15.107:0/1731373545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 0x7fb440101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 -- 172.21.15.107:0/1731373545 >> 172.21.15.107:0/1731373545 conn(0x7fb4400fcf00 msgr2=0x7fb4400ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 -- 172.21.15.107:0/1731373545 shutdown_connections 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.099+0000 7fb445e23640 1 -- 172.21.15.107:0/1731373545 wait complete. 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 Processor -- start 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 -- start start 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 0x7fb44006bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb44019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010d160 con 0x7fb4401027b0 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010cfe0 con 0x7fb440103170 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb445e23640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb44010d2e0 con 0x7fb440101650 2024-08-20T06:36:37.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34286/0 (socket says 172.21.15.107:34286) 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 -- 172.21.15.107:0/1714703888 learned_addr learned my addr 172.21.15.107:0/1714703888 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb43f7fe640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 0x7fb44006bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 -- 172.21.15.107:0/1714703888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 msgr2=0x7fb44006bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 0x7fb44006bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb43ffff640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb44019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 -- 172.21.15.107:0/1714703888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 msgr2=0x7fb44019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb44019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb437fff640 1 -- 172.21.15.107:0/1714703888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4401a0610 con 0x7fb4401027b0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.100+0000 7fb43ffff640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb44019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb437fff640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb428004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42803f030 con 0x7fb4401027b0 2024-08-20T06:36:37.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb428003c90 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb428035070 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4401a08a0 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.101+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4401a0d80 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.102+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4401a0fe0 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.103+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fb428042030 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.104+0000 7fb43d7fa640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 0x7fb42c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.104+0000 7fb43f7fe640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 0x7fb42c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.104+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb4281044c0 con 0x7fb4401027b0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.104+0000 7fb43f7fe640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 0x7fb42c0b6090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4300058c0 tx=0x7fb43000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:37.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.108+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb4280cbb10 con 0x7fb4401027b0 2024-08-20T06:36:37.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.296+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb4401042a0 con 0x7fb42c0b3bd0 2024-08-20T06:36:37.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.298+0000 7fb43d7fa640 1 -- 172.21.15.107:0/1714703888 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb4401042a0 con 0x7fb42c0b3bd0 2024-08-20T06:36:37.298 INFO:teuthology.orchestra.run.smithi107.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-20T06:36:37.299 INFO:teuthology.orchestra.run.smithi107.stdout:smithi107 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NK400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-20T06:36:37.299 INFO:teuthology.orchestra.run.smithi107.stdout:smithi183 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500QJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-20T06:36:37.299 INFO:teuthology.orchestra.run.smithi107.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 msgr2=0x7fb42c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 0x7fb42c0b6090 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb4300058c0 tx=0x7fb43000a4c0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 msgr2=0x7fb44019d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb428009230 tx=0x7fb428004270 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 shutdown_connections 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb440103170 0x7fb44019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4401027b0 0x7fb44019d760 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb42c0b3bd0 0x7fb42c0b6090 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 --2- 172.21.15.107:0/1714703888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb440101650 0x7fb44006bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 >> 172.21.15.107:0/1714703888 conn(0x7fb4400fcf00 msgr2=0x7fb440103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:37.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 shutdown_connections 2024-08-20T06:36:37.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:37.300+0000 7fb445e23640 1 -- 172.21.15.107:0/1714703888 wait complete. 2024-08-20T06:36:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:37 smithi107 ceph-mon[18495]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:37 smithi107 ceph-mon[18495]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:37 smithi183 ceph-mon[22204]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:37 smithi183 ceph-mon[22204]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:37 smithi193 ceph-mon[21944]: pgmap v320: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:37 smithi193 ceph-mon[21944]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:37.943 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T06:36:37.952 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-08-20T06:36:37.953 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-20T06:36:38.196 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:38.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 -- 172.21.15.107:0/3945732183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 msgr2=0x7efea4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:38.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 --2- 172.21.15.107:0/3945732183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4105390 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efe94009900 tx=0x7efe9402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:38.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 -- 172.21.15.107:0/3945732183 shutdown_connections 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 --2- 172.21.15.107:0/3945732183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 0x7efea410a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 --2- 172.21.15.107:0/3945732183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 0x7efea4107ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 --2- 172.21.15.107:0/3945732183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4105390 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 -- 172.21.15.107:0/3945732183 >> 172.21.15.107:0/3945732183 conn(0x7efea40fcee0 msgr2=0x7efea40ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 -- 172.21.15.107:0/3945732183 shutdown_connections 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.649+0000 7efeaafa9640 1 -- 172.21.15.107:0/3945732183 wait complete. 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 Processor -- start 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 -- start start 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 0x7efea4199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:38.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 0x7efea4199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efea410e280 con 0x7efea406b340 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efea410e100 con 0x7efea41058d0 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efeaafa9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efea410e400 con 0x7efea4108420 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efea8d45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efea8d45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34312/0 (socket says 172.21.15.107:34312) 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efea8d45640 1 -- 172.21.15.107:0/2355675695 learned_addr learned my addr 172.21.15.107:0/2355675695 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.650+0000 7efea9546640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 0x7efea4199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea3fff640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 0x7efea4199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 -- 172.21.15.107:0/2355675695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 msgr2=0x7efea4199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 0x7efea4199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 -- 172.21.15.107:0/2355675695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 msgr2=0x7efea4199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 0x7efea4199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 -- 172.21.15.107:0/2355675695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea419a530 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea8d45640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efe94009230 tx=0x7efe940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9403f030 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe94003c90 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea419a7c0 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94035070 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.651+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea41a0f00 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.652+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea4105cd0 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.654+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7efe94042030 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.655+0000 7efea1ffb640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 0x7efe8c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.655+0000 7efea3fff640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 0x7efe8c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.655+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7efe94104860 con 0x7efea406b340 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.655+0000 7efea3fff640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 0x7efe8c0b6110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe98009f20 tx=0x7efe980094f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:38.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.658+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efe940cbe70 con 0x7efea406b340 2024-08-20T06:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.869+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7efea40fe9d0 con 0x7efea406b340 2024-08-20T06:36:38.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.869+0000 7efea1ffb640 1 -- 172.21.15.107:0/2355675695 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3102 (secure 0 0 0) 0x7efea40fe9d0 con 0x7efea406b340 2024-08-20T06:36:38.870 INFO:teuthology.orchestra.run.smithi107.stdout:e24 2024-08-20T06:36:38.870 INFO:teuthology.orchestra.run.smithi107.stdout:btime 2024-08-20T06:35:11:971521+0000 2024-08-20T06:36:38.870 INFO:teuthology.orchestra.run.smithi107.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:legacy client fscid: 1 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:Filesystem 'cephfs' (1) 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:fs_name cephfs 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 24 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-08-20T06:35:00.478558+0000 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-08-20T06:35:11.548444+0000 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:tableserver 0 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:root 0 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:session_timeout 300 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:session_autoclose 300 2024-08-20T06:36:38.871 INFO:teuthology.orchestra.run.smithi107.stdout:max_file_size 1099511627776 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:max_xattr_size 65536 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:required_client_features {} 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure 0 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:last_failure_osd_epoch 0 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:max_mds 3 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:in 0,1,2 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:up {0=24572,1=24562,2=24548} 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:failed 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:damaged 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:stopped 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:data_pools [3] 2024-08-20T06:36:38.872 INFO:teuthology.orchestra.run.smithi107.stdout:metadata_pool 2 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:inline_data disabled 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:balancer 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:bal_rank_mask -1 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:standby_count_wanted 0 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:qdb_cluster leader: 24572 members: 24548,24562,24572 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.b{0:24572} state up:active seq 4 addr [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.c{0:24547} state up:standby-replay seq 1 addr [v2:172.21.15.193:6836/2519256232,v1:172.21.15.193:6837/2519256232] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.i{1:24562} state up:active seq 8 addr [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.f{1:24541} state up:standby-replay seq 1 addr [v2:172.21.15.193:6834/2974563037,v1:172.21.15.193:6835/2974563037] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.h{2:24548} state up:active seq 9 addr [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.e{2:24533} state up:standby-replay seq 1 addr [v2:172.21.15.183:6834/2773184516,v1:172.21.15.183:6835/2773184516] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:38.873 INFO:teuthology.orchestra.run.smithi107.stdout:Standby daemons: 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.j{-1:14691} state up:standby seq 1 addr [v2:172.21.15.107:6834/3097875231,v1:172.21.15.107:6835/3097875231] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.d{-1:14703} state up:standby seq 1 addr [v2:172.21.15.107:6836/3157081894,v1:172.21.15.107:6837/3157081894] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.g{-1:14757} state up:standby seq 1 addr [v2:172.21.15.107:6838/1902182493,v1:172.21.15.107:6839/1902182493] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.a{-1:14769} state up:standby seq 1 addr [v2:172.21.15.107:6840/1702418500,v1:172.21.15.107:6841/1702418500] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.l{-1:24508} state up:standby seq 1 addr [v2:172.21.15.193:6832/154109335,v1:172.21.15.193:6833/154109335] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.874 INFO:teuthology.orchestra.run.smithi107.stdout:[mds.k{-1:24512} state up:standby seq 1 addr [v2:172.21.15.183:6832/4048095343,v1:172.21.15.183:6833/4048095343] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 msgr2=0x7efe8c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 0x7efe8c0b6110 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efe98009f20 tx=0x7efe980094f0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 msgr2=0x7efea4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efe94009230 tx=0x7efe940042f0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 shutdown_connections 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe8c0b3c50 0x7efe8c0b6110 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efea4108420 0x7efea4199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efea41058d0 0x7efea4199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 --2- 172.21.15.107:0/2355675695 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efea406b340 0x7efea4198a00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 >> 172.21.15.107:0/2355675695 conn(0x7efea40fcee0 msgr2=0x7efea4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:38.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 shutdown_connections 2024-08-20T06:36:38.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:38.871+0000 7efeaafa9640 1 -- 172.21.15.107:0/2355675695 wait complete. 2024-08-20T06:36:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:39 smithi183 ceph-mon[22204]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:39 smithi183 ceph-mon[22204]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:39 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2355675695' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T06:36:39.524 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:39 smithi107 ceph-mon[18495]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:39.524 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:39 smithi107 ceph-mon[18495]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:39.524 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:39 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2355675695' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T06:36:39.550 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-20T06:36:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:39 smithi193 ceph-mon[21944]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:39 smithi193 ceph-mon[21944]: pgmap v321: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:39 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2355675695' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T06:36:39.800 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 -- 172.21.15.107:0/1100388694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 msgr2=0x7f537c102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 --2- 172.21.15.107:0/1100388694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c102bf0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f536c009930 tx=0x7f536c02a420 comp rx=0 tx=0).stop 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 -- 172.21.15.107:0/1100388694 shutdown_connections 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 --2- 172.21.15.107:0/1100388694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c103130 0x7f537c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 --2- 172.21.15.107:0/1100388694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c102bf0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 --2- 172.21.15.107:0/1100388694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c101610 0x7f537c101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 -- 172.21.15.107:0/1100388694 >> 172.21.15.107:0/1100388694 conn(0x7f537c0fcee0 msgr2=0x7f537c0ff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 -- 172.21.15.107:0/1100388694 shutdown_connections 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.276+0000 7f538333e640 1 -- 172.21.15.107:0/1100388694 wait complete. 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 Processor -- start 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 -- start start 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c101610 0x7f537c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 0x7f537c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10d140 con 0x7f537c102770 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10cfc0 con 0x7f537c101610 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f538333e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10d2c0 con 0x7f537c103130 2024-08-20T06:36:40.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34326/0 (socket says 172.21.15.107:34326) 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 -- 172.21.15.107:0/2439562683 learned_addr learned my addr 172.21.15.107:0/2439562683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53810da640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c101610 0x7f537c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53818db640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 0x7f537c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 -- 172.21.15.107:0/2439562683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 msgr2=0x7f537c199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 0x7f537c199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 -- 172.21.15.107:0/2439562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c101610 msgr2=0x7f537c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c101610 0x7f537c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53808d9640 1 -- 172.21.15.107:0/2439562683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c19a590 con 0x7f537c102770 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.277+0000 7f53818db640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 0x7f537c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f53808d9640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f536c009a50 tx=0x7f536c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c03f030 con 0x7f537c102770 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c19a820 con 0x7f537c102770 2024-08-20T06:36:40.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f536c003a80 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c035070 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.278+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a0f60 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.279+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c1a1130 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.280+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f536c042030 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.282+0000 7f53727fc640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 0x7f53680b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.282+0000 7f53810da640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 0x7f53680b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.282+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f536c1047c0 con 0x7f537c102770 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.282+0000 7f53810da640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 0x7f53680b6110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5378009bd0 tx=0x7f5378005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:40.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.284+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f536c0cbdd0 con 0x7f537c102770 2024-08-20T06:36:40.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.468+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f537c1a1300 con 0x7f537c102770 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.469+0000 7f53727fc640 1 -- 172.21.15.107:0/2439562683 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v73) ==== 56+0+5341 (secure 0 0 0) 0x7f536c0cb6d0 con 0x7f537c102770 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:epoch 73 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:created 2024-08-20T06:27:17.087848+0000 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:modified 2024-08-20T06:36:03.117597+0000 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-20T06:36:40.470 INFO:teuthology.orchestra.run.smithi107.stdout:crush_version 26 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:full_ratio 0.95 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:backfillfull_ratio 0.9 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:nearfull_ratio 0.85 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:require_min_compat_client luminous 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:min_compat_client luminous 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:require_osd_release squid 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:stretch_mode_enabled false 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 20 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 71 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 71 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:max_osd 12 2024-08-20T06:36:40.471 INFO:teuthology.orchestra.run.smithi107.stdout:osd.0 up in weight 1 up_from 8 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6802/3184473884,v1:172.21.15.107:6803/3184473884] [v2:172.21.15.107:6804/3184473884,v1:172.21.15.107:6805/3184473884] exists,up 8fb671ab-2dd3-4934-9bf5-4492165e868c 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.1 up in weight 1 up_from 12 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6800/2162362081,v1:172.21.15.183:6801/2162362081] [v2:172.21.15.183:6802/2162362081,v1:172.21.15.183:6803/2162362081] exists,up 7896e40d-68d0-4ca3-9a69-5364d3804903 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.2 up in weight 1 up_from 17 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6800/4087892550,v1:172.21.15.193:6801/4087892550] [v2:172.21.15.193:6802/4087892550,v1:172.21.15.193:6803/4087892550] exists,up 371c31f4-04e5-4bce-87c7-b8db82e15d1c 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.3 up in weight 1 up_from 23 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6810/3836170124,v1:172.21.15.107:6811/3836170124] [v2:172.21.15.107:6812/3836170124,v1:172.21.15.107:6813/3836170124] exists,up 89b8f09d-16d2-4c69-8d1e-d2835a148199 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.4 up in weight 1 up_from 29 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6808/3081917594,v1:172.21.15.183:6809/3081917594] [v2:172.21.15.183:6810/3081917594,v1:172.21.15.183:6811/3081917594] exists,up 1047c4ef-e210-49ca-86d1-84d0b9af15e2 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.5 up in weight 1 up_from 34 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6808/2859012994,v1:172.21.15.193:6809/2859012994] [v2:172.21.15.193:6810/2859012994,v1:172.21.15.193:6811/2859012994] exists,up 6e114967-5347-4fb6-a53b-db12e51cd2c1 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.6 up in weight 1 up_from 39 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6818/2830948691,v1:172.21.15.107:6819/2830948691] [v2:172.21.15.107:6820/2830948691,v1:172.21.15.107:6821/2830948691] exists,up 56c0c67a-ffc9-49e3-976f-4a1c19cc6786 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.7 up in weight 1 up_from 45 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6816/3897620215,v1:172.21.15.183:6817/3897620215] [v2:172.21.15.183:6818/3897620215,v1:172.21.15.183:6819/3897620215] exists,up 2ed828c5-26a6-488d-a45a-18937c0520cb 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.8 up in weight 1 up_from 51 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6816/2908108197,v1:172.21.15.193:6817/2908108197] [v2:172.21.15.193:6818/2908108197,v1:172.21.15.193:6819/2908108197] exists,up 1f41a3e6-90c8-46c8-bcd6-883e5b7a253a 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.9 up in weight 1 up_from 56 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.107:6826/1423000516,v1:172.21.15.107:6827/1423000516] [v2:172.21.15.107:6828/1423000516,v1:172.21.15.107:6829/1423000516] exists,up ec582cc0-f65e-403c-bd4f-ab7077dd1f38 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.10 up in weight 1 up_from 60 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.183:6824/2667647347,v1:172.21.15.183:6825/2667647347] [v2:172.21.15.183:6826/2667647347,v1:172.21.15.183:6827/2667647347] exists,up 2a94743b-6ba0-4a13-878b-a5e931783b7d 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:osd.11 up in weight 1 up_from 65 up_thru 72 down_at 0 last_clean_interval [0,0) [v2:172.21.15.193:6824/735254759,v1:172.21.15.193:6825/735254759] [v2:172.21.15.193:6826/735254759,v1:172.21.15.193:6827/735254759] exists,up e25cf09b-b923-401d-9d8b-16f2cfff57bd 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.0 [8,1] 2024-08-20T06:36:40.472 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.5 [8,11] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.13 [2,11] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 2.14 [2,4] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.c [3,9] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.13 [4,9] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.17 [4,6] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.39 [5,8] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:pg_upmap_items 3.3c [4,9] 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/1518535638 expires 2024-08-21T06:28:01.208606+0000 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/2483940460 expires 2024-08-21T06:28:01.208606+0000 2024-08-20T06:36:40.473 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3373920322 expires 2024-08-21T06:28:01.208606+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/3759743791 expires 2024-08-21T06:28:01.208606+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/3013940704 expires 2024-08-21T06:27:38.122407+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6801/235474616 expires 2024-08-21T06:27:38.122407+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/235474616 expires 2024-08-21T06:27:38.122407+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/484222332 expires 2024-08-21T06:27:38.122407+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:6800/3759743791 expires 2024-08-21T06:28:01.208606+0000 2024-08-20T06:36:40.474 INFO:teuthology.orchestra.run.smithi107.stdout:blocklist 172.21.15.107:0/343023724 expires 2024-08-21T06:27:38.122407+0000 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 msgr2=0x7f53680b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 0x7f53680b6110 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5378009bd0 tx=0x7f5378005f90 comp rx=0 tx=0).stop 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 msgr2=0x7f537c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f536c009a50 tx=0x7f536c004270 comp rx=0 tx=0).stop 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 shutdown_connections 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f53680b3c50 0x7f53680b6110 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f537c103130 0x7f537c199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f537c102770 0x7f537c1990b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 --2- 172.21.15.107:0/2439562683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f537c101610 0x7f537c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:40.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 >> 172.21.15.107:0/2439562683 conn(0x7f537c0fcee0 msgr2=0x7f537c0fea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:40.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 shutdown_connections 2024-08-20T06:36:40.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:40.471+0000 7f538333e640 1 -- 172.21.15.107:0/2439562683 wait complete. 2024-08-20T06:36:41.111 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-20T06:36:41.122 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:41.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:41 smithi107 ceph-mon[18495]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:41.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:41 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2439562683' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T06:36:41.369 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:41 smithi183 ceph-mon[22204]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:41 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2439562683' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T06:36:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:41 smithi193 ceph-mon[21944]: pgmap v322: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-08-20T06:36:41.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:41 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2439562683' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 -- 172.21.15.107:0/1384662253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac41084e0 msgr2=0x7f5ac4108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 --2- 172.21.15.107:0/1384662253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac41084e0 0x7f5ac4108920 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009900 tx=0x7f5ab802a410 comp rx=0 tx=0).stop 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 -- 172.21.15.107:0/1384662253 shutdown_connections 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 --2- 172.21.15.107:0/1384662253 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac4108e60 0x7f5ac410f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 --2- 172.21.15.107:0/1384662253 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac41084e0 0x7f5ac4108920 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 --2- 172.21.15.107:0/1384662253 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.840+0000 7f5acb90f640 1 -- 172.21.15.107:0/1384662253 >> 172.21.15.107:0/1384662253 conn(0x7f5ac40fe540 msgr2=0x7f5ac4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- 172.21.15.107:0/1384662253 shutdown_connections 2024-08-20T06:36:41.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- 172.21.15.107:0/1384662253 wait complete. 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 Processor -- start 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- start start 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 0x7f5ac4199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41129d0 con 0x7f5ac4108e60 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112850 con 0x7f5ac4102ad0 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5acb90f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112b50 con 0x7f5ac41084e0 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5ac9eac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5ac9eac640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34338/0 (socket says 172.21.15.107:34338) 2024-08-20T06:36:41.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5ac9eac640 1 -- 172.21.15.107:0/2643318525 learned_addr learned my addr 172.21.15.107:0/2643318525 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.841+0000 7f5ac96ab640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac8eaa640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 0x7f5ac4199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 -- 172.21.15.107:0/2643318525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 msgr2=0x7f5ac4199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 0x7f5ac4199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 -- 172.21.15.107:0/2643318525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 msgr2=0x7f5ac4198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 -- 172.21.15.107:0/2643318525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac419f070 con 0x7f5ac4108e60 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac96ab640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac8eaa640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 0x7f5ac4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ac9eac640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cad0 tx=0x7f5abc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00ae20 con 0x7f5ac4108e60 2024-08-20T06:36:41.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5abc005020 con 0x7f5ac4108e60 2024-08-20T06:36:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac419f360 con 0x7f5ac4108e60 2024-08-20T06:36:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00f6b0 con 0x7f5ac4108e60 2024-08-20T06:36:41.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.842+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41a5a20 con 0x7f5ac4108e60 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.843+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac4102eb0 con 0x7f5ac4108e60 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.844+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f5abc0051d0 con 0x7f5ac4108e60 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.845+0000 7f5ab67fc640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 0x7f5ab00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.845+0000 7f5ac96ab640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 0x7f5ab00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.845+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5abc0e3ff0 con 0x7f5ac4108e60 2024-08-20T06:36:41.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.846+0000 7f5ac96ab640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 0x7f5ab00b6110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009930 tx=0x7f5ac0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:41.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:41.849+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5abc014030 con 0x7f5ac4108e60 2024-08-20T06:36:42.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.063+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ac402e340 con 0x7f5ac4108e60 2024-08-20T06:36:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.064+0000 7f5ab67fc640 1 -- 172.21.15.107:0/2643318525 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7f5abc0ab640 con 0x7f5ac4108e60 2024-08-20T06:36:42.065 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:42.065 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:36:42.065 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:42.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 msgr2=0x7f5ab00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:42.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 0x7f5ab00b6110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009930 tx=0x7f5ac0008040 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 msgr2=0x7f5ac419e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00cad0 tx=0x7f5abc00cf90 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 shutdown_connections 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5ac4108e60 0x7f5ac419e8a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5ab00b3c50 0x7f5ab00b6110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ac41084e0 0x7f5ac4199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.066+0000 7f5acb90f640 1 --2- 172.21.15.107:0/2643318525 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5ac4102ad0 0x7f5ac4198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.067+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 >> 172.21.15.107:0/2643318525 conn(0x7f5ac40fe540 msgr2=0x7f5ac4100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.067+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 shutdown_connections 2024-08-20T06:36:42.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:42.067+0000 7f5acb90f640 1 -- 172.21.15.107:0/2643318525 wait complete. 2024-08-20T06:36:42.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:42 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2643318525' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:42.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:42 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2643318525' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:42.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:42 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2643318525' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:42.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:42.944 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:43.326 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:43 smithi107 ceph-mon[18495]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 -- 172.21.15.107:0/2902241397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601084c0 msgr2=0x7efe60108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 --2- 172.21.15.107:0/2902241397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601084c0 0x7efe60108900 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe54009900 tx=0x7efe5402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 -- 172.21.15.107:0/2902241397 shutdown_connections 2024-08-20T06:36:43.428 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 --2- 172.21.15.107:0/2902241397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe60108e40 0x7efe6010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 --2- 172.21.15.107:0/2902241397 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe601084c0 0x7efe60108900 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 --2- 172.21.15.107:0/2902241397 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 0x7efe60102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 -- 172.21.15.107:0/2902241397 >> 172.21.15.107:0/2902241397 conn(0x7efe600fe500 msgr2=0x7efe60100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 -- 172.21.15.107:0/2902241397 shutdown_connections 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.403+0000 7efe65d11640 1 -- 172.21.15.107:0/2902241397 wait complete. 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 Processor -- start 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 -- start start 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 0x7efe6019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe601084c0 0x7efe6019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efe60112940 con 0x7efe60108e40 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efe601127c0 con 0x7efe60102ab0 2024-08-20T06:36:43.429 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe65d11640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efe60112ac0 con 0x7efe601084c0 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe5ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34362/0 (socket says 172.21.15.107:34362) 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.404+0000 7efe5ffff640 1 -- 172.21.15.107:0/717939025 learned_addr learned my addr 172.21.15.107:0/717939025 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5f7fe640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 0x7efe6019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5effd640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe601084c0 0x7efe6019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 -- 172.21.15.107:0/717939025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe601084c0 msgr2=0x7efe6019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe601084c0 0x7efe6019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 -- 172.21.15.107:0/717939025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 msgr2=0x7efe6019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 0x7efe6019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 -- 172.21.15.107:0/717939025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe601a1120 con 0x7efe60108e40 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5ffff640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe5800cad0 tx=0x7efe5800ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:43.430 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5800ad60 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efe58005020 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe601a1410 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5800f6b0 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.405+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe601a7ad0 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.406+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe60102e90 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.407+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7efe580051d0 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.408+0000 7efe5cff9640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 0x7efe480b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.408+0000 7efe5f7fe640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 0x7efe480b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.408+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7efe580e3ae0 con 0x7efe60108e40 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.409+0000 7efe5f7fe640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 0x7efe480b6110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe5000abe0 tx=0x7efe5000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:43.431 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.412+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efe58014030 con 0x7efe60108e40 2024-08-20T06:36:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:43 smithi183 ceph-mon[22204]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:43 smithi193 ceph-mon[21944]: pgmap v323: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-08-20T06:36:43.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.624+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe6010f060 con 0x7efe60108e40 2024-08-20T06:36:43.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.624+0000 7efe5cff9640 1 -- 172.21.15.107:0/717939025 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11935 (secure 0 0 0) 0x7efe580ab130 con 0x7efe60108e40 2024-08-20T06:36:43.626 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:43.626 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":24,"btime":"2024-08-20T06:35:11:971521+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:35:11.548444+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:36:43.626 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 24 2024-08-20T06:36:43.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 msgr2=0x7efe480b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:43.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 0x7efe480b6110 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efe5000abe0 tx=0x7efe5000a2d0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 msgr2=0x7efe601a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe5800cad0 tx=0x7efe5800ced0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 shutdown_connections 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efe60108e40 0x7efe601a0950 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efe480b3c50 0x7efe480b6110 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe601084c0 0x7efe6019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 --2- 172.21.15.107:0/717939025 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efe60102ab0 0x7efe6019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.626+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 >> 172.21.15.107:0/717939025 conn(0x7efe600fe500 msgr2=0x7efe60105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.627+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 shutdown_connections 2024-08-20T06:36:43.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:43.627+0000 7efe65d11640 1 -- 172.21.15.107:0/717939025 wait complete. 2024-08-20T06:36:44.302 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:36:44.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:44 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/717939025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:44.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:44 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/717939025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:44.550 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:44.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:44 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/717939025' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 -- 172.21.15.107:0/3459277468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 msgr2=0x7f3b78101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/3459277468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b78101900 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009900 tx=0x7f3b6c02a410 comp rx=0 tx=0).stop 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 -- 172.21.15.107:0/3459277468 shutdown_connections 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/3459277468 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b78111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/3459277468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7810ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/3459277468 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b78101900 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 -- 172.21.15.107:0/3459277468 >> 172.21.15.107:0/3459277468 conn(0x7f3b78076c80 msgr2=0x7f3b78077090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 -- 172.21.15.107:0/3459277468 shutdown_connections 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.995+0000 7f3b7eab7640 1 -- 172.21.15.107:0/3459277468 wait complete. 2024-08-20T06:36:45.034 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 Processor -- start 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 -- start start 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b781a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114ba0 con 0x7f3b78101520 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114a20 con 0x7f3b7810f000 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7eab7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b78114d20 con 0x7f3b78101e40 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34380/0 (socket says 172.21.15.107:34380) 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 -- 172.21.15.107:0/2655421108 learned_addr learned my addr 172.21.15.107:0/2655421108 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b77fff640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:45.035 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7d054640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b781a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 -- 172.21.15.107:0/2655421108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 msgr2=0x7f3b7819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 -- 172.21.15.107:0/2655421108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 msgr2=0x7f3b781a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b781a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 -- 172.21.15.107:0/2655421108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b781a1240 con 0x7f3b78101520 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b77fff640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7d054640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b781a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.996+0000 7f3b7c853640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009a20 tx=0x7f3b6c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.997+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c03f030 con 0x7f3b78101520 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.997+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b781a14d0 con 0x7f3b78101520 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.997+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b6c0035f0 con 0x7f3b78101520 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.997+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c035070 con 0x7f3b78101520 2024-08-20T06:36:45.036 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.997+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b781a7bf0 con 0x7f3b78101520 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.998+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b78101900 con 0x7f3b78101520 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:44.999+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f3b6c003aa0 con 0x7f3b78101520 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.000+0000 7f3b75ffb640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 0x7f3b680b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.000+0000 7f3b77fff640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 0x7f3b680b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.000+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3b6c104c40 con 0x7f3b78101520 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.000+0000 7f3b77fff640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 0x7f3b680b6090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:45.037 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.003+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3b6c0cc290 con 0x7f3b78101520 2024-08-20T06:36:45.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.180+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b7819c2c0 con 0x7f3b78101520 2024-08-20T06:36:45.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.182+0000 7f3b75ffb640 1 -- 172.21.15.107:0/2655421108 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7f3b6c0a5e20 con 0x7f3b78101520 2024-08-20T06:36:45.183 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:45.183 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:36:45.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 msgr2=0x7f3b680b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:45.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 0x7f3b680b6090 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b64006fd0 tx=0x7f3b64008040 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 msgr2=0x7f3b7819b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c009a20 tx=0x7f3b6c004270 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 shutdown_connections 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b7810f000 0x7f3b781a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b680b3bd0 0x7f3b680b6090 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b78101e40 0x7f3b7819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 --2- 172.21.15.107:0/2655421108 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b78101520 0x7f3b7819b190 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 >> 172.21.15.107:0/2655421108 conn(0x7f3b78076c80 msgr2=0x7f3b7810f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.184+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 shutdown_connections 2024-08-20T06:36:45.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:45.185+0000 7f3b7eab7640 1 -- 172.21.15.107:0/2655421108 wait complete. 2024-08-20T06:36:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:45 smithi183 ceph-mon[22204]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:45 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2655421108' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:45.516 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:45 smithi107 ceph-mon[18495]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:45.516 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:45 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2655421108' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:45 smithi193 ceph-mon[21944]: pgmap v324: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-08-20T06:36:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:45 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2655421108' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:45.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T06:35:00.478558+0000', 'modified': '2024-08-20T06:35:11.548444+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24572, 'mds_1': 24562, 'mds_2': 24548}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24533': {'gid': 24533, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.183:6835/2773184516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6834', 'nonce': 2773184516}, {'type': 'v1', 'addr': '172.21.15.183:6835', 'nonce': 2773184516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24541': {'gid': 24541, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6835/2974563037', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 2974563037}, {'type': 'v1', 'addr': '172.21.15.193:6835', 'nonce': 2974563037}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.193:6837/2519256232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 2519256232}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2519256232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24548': {'gid': 24548, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.183:6837/2933033317', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6836', 'nonce': 2933033317}, {'type': 'v1', 'addr': '172.21.15.183:6837', 'nonce': 2933033317}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24562': {'gid': 24562, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.193:6839/3581594961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6838', 'nonce': 3581594961}, {'type': 'v1', 'addr': '172.21.15.193:6839', 'nonce': 3581594961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24572': {'gid': 24572, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.183:6839/2078682144', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6838', 'nonce': 2078682144}, {'type': 'v1', 'addr': '172.21.15.183:6839', 'nonce': 2078682144}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24572, 'qdb_cluster': [24548, 24562, 24572]} 2024-08-20T06:36:45.960 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-20T06:36:45.960 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T06:36:46.206 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- 172.21.15.107:0/4053342426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4701073a0 msgr2=0x7fb4701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 --2- 172.21.15.107:0/4053342426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4701073a0 0x7fb4701077e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb460009900 tx=0x7fb46002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:46.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- 172.21.15.107:0/4053342426 shutdown_connections 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 --2- 172.21.15.107:0/4053342426 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb470107d20 0x7fb47010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 --2- 172.21.15.107:0/4053342426 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb4701073a0 0x7fb4701077e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 --2- 172.21.15.107:0/4053342426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb470101990 0x7fb470101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- 172.21.15.107:0/4053342426 >> 172.21.15.107:0/4053342426 conn(0x7fb4700fd400 msgr2=0x7fb4700ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- 172.21.15.107:0/4053342426 shutdown_connections 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- 172.21.15.107:0/4053342426 wait complete. 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 Processor -- start 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.816+0000 7fb4772d7640 1 -- start start 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 0x7fb4701995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb470107d20 0x7fb47019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb4701129d0 con 0x7fb470101990 2024-08-20T06:36:46.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112850 con 0x7fb4701073a0 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112b50 con 0x7fb470107d20 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36008/0 (socket says 172.21.15.107:36008) 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 -- 172.21.15.107:0/924379606 learned_addr learned my addr 172.21.15.107:0/924379606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb474872640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 0x7fb4701995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 -- 172.21.15.107:0/924379606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb470107d20 msgr2=0x7fb47019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb470107d20 0x7fb47019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 -- 172.21.15.107:0/924379606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 msgr2=0x7fb4701995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 0x7fb4701995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 -- 172.21.15.107:0/924379606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb47019f130 con 0x7fb470101990 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb474872640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 0x7fb4701995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb475073640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00b4a0 tx=0x7fb46c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4667fc640 1 -- 172.21.15.107:0/924379606 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c014070 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb47019f420 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4667fc640 1 -- 172.21.15.107:0/924379606 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb46c010aa0 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4667fc640 1 -- 172.21.15.107:0/924379606 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c017b70 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.817+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4701a5b60 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 Processor -- start 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4667fc640 1 -- 172.21.15.107:0/924379606 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fb46c017070 con 0x7fb470101990 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 -- start start 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a62fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a539d96a0 con 0x563a53a63510 2024-08-20T06:36:46.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a539c8ba0 con 0x563a53a62bf0 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb4796d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a539ccbe0 con 0x563a53a6f4a0 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb475073640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb475073640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36010/0 (socket says 172.21.15.107:36010) 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb475073640 1 -- 172.21.15.107:0/3671953722 learned_addr learned my addr 172.21.15.107:0/3671953722 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb474872640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.820+0000 7fb475874640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a62fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 -- 172.21.15.107:0/3671953722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 msgr2=0x563a53a71890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a71890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 -- 172.21.15.107:0/3671953722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 msgr2=0x563a53a62fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a62fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 -- 172.21.15.107:0/3671953722 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a53728540 con 0x563a53a63510 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb474872640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:46.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475073640 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0eb5b0 tx=0x7fb46c0eb6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a91c9f0a89faaf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb4667fc640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 0x7fb45c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb475874640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 0x7fb45c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb45b7fe640 1 -- 172.21.15.107:0/3671953722 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c0f7070 con 0x563a53a63510 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb4667fc640 1 -- 172.21.15.107:0/924379606 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb46c0e3360 con 0x7fb470101990 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb45b7fe640 1 -- 172.21.15.107:0/3671953722 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb46c017360 con 0x563a53a63510 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.821+0000 7fb45b7fe640 1 -- 172.21.15.107:0/3671953722 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46c0aa080 con 0x563a53a63510 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- 172.21.15.107:0/3671953722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 msgr2=0x563a53a6ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb475874640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 0x7fb45c0b6090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb468009e40 tx=0x7fb468009430 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fb46c0eb5b0 tx=0x7fb46c0eb6a0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- 172.21.15.107:0/3671953722 shutdown_connections 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a71890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6ef60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 --2- 172.21.15.107:0/3671953722 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a62fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- 172.21.15.107:0/3671953722 >> 172.21.15.107:0/3671953722 conn(0x563a53a60a20 msgr2=0x563a53a61bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- 172.21.15.107:0/3671953722 shutdown_connections 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- 172.21.15.107:0/3671953722 wait complete. 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 -- start start 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.822+0000 7fb4796d0740 1 Processor -- start 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a6c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a5382b5f0 con 0x563a53a63510 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a5394f6c0 con 0x563a53a62bf0 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a53a72150 con 0x563a53a6f4a0 2024-08-20T06:36:46.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb4796d0740 10 client.0 _ll_register_callbacks cb 0x563a5398c1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36024/0 (socket says 172.21.15.107:36024) 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 -- 172.21.15.107:0/4097021302 learned_addr learned my addr 172.21.15.107:0/4097021302 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb475073640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a6c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb475874640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 -- 172.21.15.107:0/4097021302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 msgr2=0x563a53a6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a6cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 -- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 msgr2=0x563a53a6c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a6c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 -- 172.21.15.107:0/4097021302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a53a7b530 con 0x563a53a63510 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb474872640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4600057c0 tx=0x7fb460031770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea2f6acdf589555 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.823+0000 7fb458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:46.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46003f070 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb460004450 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563a539db5f0 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb460035070 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 10 client.15273 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563a53a7dee0 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563a539db5f0 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb460004b80 con 0x563a53a63510 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 10 client.15273 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 10 client.15273 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 20 client.15273 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53a80500 0x563a53a81970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 4 client.15273 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a539d7090 con 0x563a53a80500 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.824+0000 7fb475073640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53a80500 0x563a53a81970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.825+0000 7fb475073640 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53a80500 0x563a53a81970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1aeec83079770ccc server_cookie=f78f20b6cd20e9f8 in_seq=0 out_seq=0 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.825+0000 7fb458ff9640 10 client.15273 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.825+0000 7fb458ff9640 1 -- 172.21.15.107:0/4097021302 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563a539d7090 con 0x563a53a80500 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb458ff9640 10 client.15273 handle_command_reply: tid=0 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 client.15273 shutdown 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 msgr2=0x563a53a6c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb4600057c0 tx=0x7fb460031770 comp rx=0 tx=0).stop 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 shutdown_connections 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a6f4a0 0x563a53a6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6c9b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53a80500 0x563a53a81970 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 --2- 172.21.15.107:0/4097021302 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a62bf0 0x563a53a6c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb458ff9640 0 client.15273 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 >> 172.21.15.107:0/4097021302 conn(0x563a53a60a20 msgr2=0x563a53a665c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 shutdown_connections 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 1 -- 172.21.15.107:0/4097021302 wait complete. 2024-08-20T06:36:46.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.826+0000 7fb4796d0740 20 client.15273 trim_cache size 0 max 16384 2024-08-20T06:36:46.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 Processor -- start 2024-08-20T06:36:46.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 -- start start 2024-08-20T06:36:46.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 0x563a53a61a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a53a7fcb0 con 0x563a53a63510 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a53a80380 con 0x563a53a6f4a0 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb4796d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a53a80a50 con 0x563a53a62bf0 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36040/0 (socket says 172.21.15.107:36040) 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 -- 172.21.15.107:0/1440550423 learned_addr learned my addr 172.21.15.107:0/1440550423 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb475874640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 0x563a53a61a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb475073640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 -- 172.21.15.107:0/1440550423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 msgr2=0x563a53a66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a66db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 -- 172.21.15.107:0/1440550423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 msgr2=0x563a53a61a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 0x563a53a61a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 -- 172.21.15.107:0/1440550423 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a53a614a0 con 0x563a53a63510 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb475874640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 0x563a53a61a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb475073640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb474872640 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb460009850 tx=0x7fb46002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc6745f7ceb46d2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.844+0000 7fb45affd640 1 -- 172.21.15.107:0/1440550423 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb46003f070 con 0x563a53a63510 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb45affd640 1 -- 172.21.15.107:0/1440550423 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4600039b0 con 0x563a53a63510 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb45affd640 1 -- 172.21.15.107:0/1440550423 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4600044b0 con 0x563a53a63510 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- 172.21.15.107:0/1440550423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 msgr2=0x563a53a6b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb460009850 tx=0x7fb46002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- 172.21.15.107:0/1440550423 shutdown_connections 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a6f4a0 0x563a53a61a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a63510 0x563a53a6b960 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 --2- 172.21.15.107:0/1440550423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- 172.21.15.107:0/1440550423 >> 172.21.15.107:0/1440550423 conn(0x563a53a60a20 msgr2=0x563a53a6acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- 172.21.15.107:0/1440550423 shutdown_connections 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- 172.21.15.107:0/1440550423 wait complete. 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.845+0000 7fb4796d0740 1 -- start start 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 Processor -- start 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a87a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 0x563a53a87fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563a53a817f0 con 0x563a53a6f4a0 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563a53a81120 con 0x563a53a63510 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563a53a81ec0 con 0x563a53a62bf0 2024-08-20T06:36:46.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb4796d0740 10 client.0 _ll_register_callbacks cb 0x7fb460009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36056/0 (socket says 172.21.15.107:36056) 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 -- 172.21.15.107:0/2142964832 learned_addr learned my addr 172.21.15.107:0/2142964832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475073640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a87a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb474872640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 0x563a53a87fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 -- 172.21.15.107:0/2142964832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 msgr2=0x563a53a87a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a87a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 -- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 msgr2=0x563a53a87fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 0x563a53a87fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475874640 1 -- 172.21.15.107:0/2142964832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a53a839b0 con 0x563a53a6f4a0 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb475073640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a87a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.846+0000 7fb474872640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 0x563a53a87fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:46.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb475874640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb468017d90 tx=0x7fb468004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9c052745becf12 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb468016ab0 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb468016c50 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4600039b0 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb468024480 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 10 client.15279 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563a53a85070 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb4600039b0 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb468022430 con 0x563a53a6f4a0 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 10 client.15279 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 10 client.15279 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 20 client.15279 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:46.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53aef230 0x563a53aedf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 4 client.15279 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.847+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "status"}) -- 0x563a539d47d0 con 0x563a53aef230 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb475073640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53aef230 0x563a53aedf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb475073640 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53aef230 0x563a53aedf40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92495969d1f7783d server_cookie=141f7b436d2d824e in_seq=0 out_seq=0 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb458ff9640 10 client.15279 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb458ff9640 1 -- 172.21.15.107:0/2142964832 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563a539d47d0 con 0x563a53aef230 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb458ff9640 10 client.15279 handle_command_reply: tid=0 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 client.15279 shutdown 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 msgr2=0x563a53a83180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb468017d90 tx=0x7fb468004b50 comp rx=0 tx=0).stop 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 shutdown_connections 2024-08-20T06:36:46.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563a53a6f4a0 0x563a53a83180 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563a53aef230 0x563a53aedf40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563a53a63510 0x563a53a87fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 --2- 172.21.15.107:0/2142964832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a53a62bf0 0x563a53a87a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 >> 172.21.15.107:0/2142964832 conn(0x563a53a60a20 msgr2=0x563a53a61a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb458ff9640 0 client.15279 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 shutdown_connections 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.848+0000 7fb4796d0740 1 -- 172.21.15.107:0/2142964832 wait complete. 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4796d0740 20 client.15279 trim_cache size 0 max 16384 2024-08-20T06:36:46.867 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 2, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24548, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 97.885337098999997, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 128.18363212899999 2024-08-20T06:36:46.868 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 msgr2=0x7fb45c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 0x7fb45c0b6090 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb468009e40 tx=0x7fb468009430 comp rx=0 tx=0).stop 2024-08-20T06:36:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 msgr2=0x7fb470199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb46c00b4a0 tx=0x7fb46c00b960 comp rx=0 tx=0).stop 2024-08-20T06:36:46.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 shutdown_connections 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb45c0b3bd0 0x7fb45c0b6090 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb470107d20 0x7fb47019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb4701073a0 0x7fb4701995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 --2- 172.21.15.107:0/924379606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb470101990 0x7fb470199080 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 >> 172.21.15.107:0/924379606 conn(0x7fb4700fd400 msgr2=0x7fb4700ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 shutdown_connections 2024-08-20T06:36:46.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:46.849+0000 7fb4772d7640 1 -- 172.21.15.107:0/924379606 wait complete. 2024-08-20T06:36:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:47 smithi183 ceph-mon[22204]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T06:36:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:47 smithi193 ceph-mon[21944]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T06:36:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:47 smithi107 ceph-mon[18495]: pgmap v325: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T06:36:47.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.i status 2024-08-20T06:36:47.943 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:48.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 -- 172.21.15.107:0/1985301167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc1084e0 msgr2=0x7f4acc108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 --2- 172.21.15.107:0/1985301167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc1084e0 0x7f4acc108920 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009900 tx=0x7f4ab402a410 comp rx=0 tx=0).stop 2024-08-20T06:36:48.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 -- 172.21.15.107:0/1985301167 shutdown_connections 2024-08-20T06:36:48.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 --2- 172.21.15.107:0/1985301167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc108e60 0x7f4acc10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 --2- 172.21.15.107:0/1985301167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc1084e0 0x7f4acc108920 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 --2- 172.21.15.107:0/1985301167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 0x7f4acc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 -- 172.21.15.107:0/1985301167 >> 172.21.15.107:0/1985301167 conn(0x7f4acc0fe540 msgr2=0x7f4acc100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 -- 172.21.15.107:0/1985301167 shutdown_connections 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 -- 172.21.15.107:0/1985301167 wait complete. 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.429+0000 7f4ad2094640 1 Processor -- start 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 -- start start 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 0x7f4acc198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc1084e0 0x7f4acc1993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1129d0 con 0x7f4acc108e60 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112850 con 0x7f4acc1084e0 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4ad2094640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112b50 con 0x7f4acc102ad0 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36076/0 (socket says 172.21.15.107:36076) 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 -- 172.21.15.107:0/883858174 learned_addr learned my addr 172.21.15.107:0/883858174 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 0x7f4acc198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acaffd640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc1084e0 0x7f4acc1993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 -- 172.21.15.107:0/883858174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 msgr2=0x7f4acc198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 0x7f4acc198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 -- 172.21.15.107:0/883858174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc1084e0 msgr2=0x7f4acc1993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc1084e0 0x7f4acc1993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.430+0000 7f4acbfff640 1 -- 172.21.15.107:0/883858174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc19ef50 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4acbfff640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b970 tx=0x7f4ac400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4ac8ff9640 1 -- 172.21.15.107:0/883858174 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4019070 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc19f240 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4ac8ff9640 1 -- 172.21.15.107:0/883858174 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ac4014440 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4ac8ff9640 1 -- 172.21.15.107:0/883858174 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40075d0 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.431+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a5980 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.433+0000 7f4ac8ff9640 1 -- 172.21.15.107:0/883858174 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f4ac401c070 con 0x7f4acc108e60 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.433+0000 7f4ad448d740 1 Processor -- start 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.433+0000 7f4ad448d740 1 -- start start 2024-08-20T06:36:48.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54130bf0 0x555e54130fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e5413f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e540a76a0 con 0x555e54131510 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e54096ba0 con 0x555e5413d4a0 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ad448d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e5409abe0 con 0x555e54130bf0 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36080/0 (socket says 172.21.15.107:36080) 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 -- 172.21.15.107:0/2665725849 learned_addr learned my addr 172.21.15.107:0/2665725849 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ac8ff9640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 0x7f4ab80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acaffd640 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54130bf0 0x555e54130fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acaffd640 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e5413f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acbfff640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 0x7f4ab80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 -- 172.21.15.107:0/2665725849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54130bf0 msgr2=0x555e54130fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54130bf0 0x555e54130fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 -- 172.21.15.107:0/2665725849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 msgr2=0x555e5413f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e5413f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 -- 172.21.15.107:0/2665725849 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e53df6540 con 0x555e54131510 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ad30 tx=0x7f4abc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b6144cb309136f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ac8ff9640 1 -- 172.21.15.107:0/883858174 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4ac40e3180 con 0x7f4acc108e60 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.434+0000 7f4ac1ffb640 1 -- 172.21.15.107:0/2665725849 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc010630 con 0x555e54131510 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4acbfff640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 0x7f4ab80b6090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40a4720 tx=0x7f4ac40d9db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ac1ffb640 1 -- 172.21.15.107:0/2665725849 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4abc014070 con 0x555e54131510 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ac1ffb640 1 -- 172.21.15.107:0/2665725849 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0189b0 con 0x555e54131510 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 -- 172.21.15.107:0/2665725849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 msgr2=0x555e5413cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00ad30 tx=0x7f4abc00f790 comp rx=0 tx=0).stop 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 -- 172.21.15.107:0/2665725849 shutdown_connections 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e5413f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54131510 0x555e5413cf60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 --2- 172.21.15.107:0/2665725849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54130bf0 0x555e54130fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 -- 172.21.15.107:0/2665725849 >> 172.21.15.107:0/2665725849 conn(0x555e5412ea20 msgr2=0x555e5412fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 -- 172.21.15.107:0/2665725849 shutdown_connections 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.435+0000 7f4ad448d740 1 -- 172.21.15.107:0/2665725849 wait complete. 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 -- start start 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 Processor -- start 2024-08-20T06:36:48.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e5413a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5413ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e53ef95f0 con 0x555e54130bf0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e5401d6c0 con 0x555e54131510 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e54140150 con 0x555e5413d4a0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4ad448d740 10 client.0 _ll_register_callbacks cb 0x555e5405a1e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36092/0 (socket says 172.21.15.107:36092) 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 -- 172.21.15.107:0/4170986599 learned_addr learned my addr 172.21.15.107:0/4170986599 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e5413a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5413ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 -- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 msgr2=0x555e5413ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5413ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 -- 172.21.15.107:0/4170986599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 msgr2=0x555e5413a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e5413a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.436+0000 7f4acaffd640 1 -- 172.21.15.107:0/4170986599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e54149480 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4acaffd640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ab402a930 tx=0x7f4ab4032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9daf86806752e1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035040 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4ab40035f0 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc014070 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ab4032e20 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 10 client.15291 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555e5414bde0 con 0x555e54130bf0 2024-08-20T06:36:48.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4abc014070 con 0x555e54130bf0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab403d980 con 0x555e54130bf0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 10 client.15291 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 20 client.15291 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e5414e3f0 0x555e5414f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 4 client.15291 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.437+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e540a95f0 con 0x555e5414e3f0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.438+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e5414e3f0 0x555e5414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.438+0000 7f4aaaffd640 10 client.15291 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.438+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e5414e3f0 0x555e5414f860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fcf990ead54ffa8e server_cookie=fd6349da0d81feb6 in_seq=0 out_seq=0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4aaaffd640 1 -- 172.21.15.107:0/4170986599 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e540a95f0 con 0x555e5414e3f0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4aaaffd640 10 client.15291 handle_command_reply: tid=0 2024-08-20T06:36:48.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 client.15291 shutdown 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 msgr2=0x555e5413cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4ab402a930 tx=0x7f4ab4032b80 comp rx=0 tx=0).stop 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 shutdown_connections 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e5414e3f0 0x555e5414f860 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5413ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e5413a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 --2- 172.21.15.107:0/4170986599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e5413cf20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4aaaffd640 0 client.15291 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 >> 172.21.15.107:0/4170986599 conn(0x555e5412ea20 msgr2=0x555e541345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 shutdown_connections 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 1 -- 172.21.15.107:0/4170986599 wait complete. 2024-08-20T06:36:48.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.439+0000 7f4ad448d740 20 client.15291 trim_cache size 0 max 16384 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.457+0000 7f4ad448d740 1 Processor -- start 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.457+0000 7f4ad448d740 1 -- start start 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e54157f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5412fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e5414dba0 con 0x555e54130bf0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e5414e270 con 0x555e54131510 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ad448d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e5414e940 con 0x555e5413d4a0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36100/0 (socket says 172.21.15.107:36100) 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 -- 172.21.15.107:0/513683272 learned_addr learned my addr 172.21.15.107:0/513683272 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acb7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e54157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5412fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 -- 172.21.15.107:0/513683272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 msgr2=0x555e5412fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5412fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 -- 172.21.15.107:0/513683272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 msgr2=0x555e54157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e54157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 -- 172.21.15.107:0/513683272 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e5412f4a0 con 0x555e54130bf0 2024-08-20T06:36:48.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4acaffd640 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40095a0 tx=0x7f4ab4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a99ec8bd4a2ada server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ac17fa640 1 -- 172.21.15.107:0/513683272 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab4035070 con 0x555e54130bf0 2024-08-20T06:36:48.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ac17fa640 1 -- 172.21.15.107:0/513683272 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ab403d8e0 con 0x555e54130bf0 2024-08-20T06:36:48.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.458+0000 7f4ac17fa640 1 -- 172.21.15.107:0/513683272 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab40038e0 con 0x555e54130bf0 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- 172.21.15.107:0/513683272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 msgr2=0x555e54134db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40095a0 tx=0x7f4ab4005c90 comp rx=0 tx=0).stop 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- 172.21.15.107:0/513683272 shutdown_connections 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e5413d4a0 0x555e5412fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e54131510 0x555e54157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 --2- 172.21.15.107:0/513683272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54134db0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- 172.21.15.107:0/513683272 >> 172.21.15.107:0/513683272 conn(0x555e5412ea20 msgr2=0x555e54135650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- 172.21.15.107:0/513683272 shutdown_connections 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- 172.21.15.107:0/513683272 wait complete. 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.459+0000 7f4ad448d740 1 -- start start 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 Processor -- start 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54131510 0x555e54156190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e541566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e5414f6e0 con 0x555e54130bf0 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e5414f010 con 0x555e5413d4a0 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e5414fdb0 con 0x555e54131510 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4ad448d740 10 client.0 _ll_register_callbacks cb 0x7f4ab4031820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:48.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36114/0 (socket says 172.21.15.107:36114) 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 -- 172.21.15.107:0/3103781720 learned_addr learned my addr 172.21.15.107:0/3103781720 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54131510 0x555e54156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e541566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 -- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54131510 msgr2=0x555e54156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54131510 0x555e54156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 -- 172.21.15.107:0/3103781720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 msgr2=0x555e541566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e541566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.460+0000 7f4acaffd640 1 -- 172.21.15.107:0/3103781720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e541bbdd0 con 0x555e54130bf0 2024-08-20T06:36:48.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4acaffd640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009900 tx=0x7f4ab4031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a9347a2cc6a47e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab406b060 con 0x555e54130bf0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4ab4003800 con 0x555e54130bf0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ab403d8e0 con 0x555e54130bf0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f4ab4035070 con 0x555e54130bf0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 10 client.15297 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555e541bd090 con 0x555e54130bf0 2024-08-20T06:36:48.468 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4ab403d8e0 con 0x555e54130bf0 2024-08-20T06:36:48.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f4ab4063030 con 0x555e54130bf0 2024-08-20T06:36:48.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:48.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 10 client.15297 resolve_mds: resolved name 'i' to daemon mds.i 2024-08-20T06:36:48.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 20 client.15297 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:48.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e541c1770 0x555e541c0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:48.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 4 client.15297 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:48.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.461+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "status"}) -- 0x555e540a5090 con 0x555e541c1770 2024-08-20T06:36:48.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e541c1770 0x555e541c0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:48.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4aaaffd640 10 client.15297 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4acb7fe640 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e541c1770 0x555e541c0480 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=920ee300d67982a7 server_cookie=cd6230619d2743d8 in_seq=0 out_seq=0 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4aaaffd640 1 -- 172.21.15.107:0/3103781720 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x555e540a5090 con 0x555e541c1770 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4aaaffd640 10 client.15297 handle_command_reply: tid=0 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 client.15297 shutdown 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 msgr2=0x555e54155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009900 tx=0x7f4ab4031750 comp rx=0 tx=0).stop 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 shutdown_connections 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x555e541c1770 0x555e541c0480 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e5413d4a0 0x555e541566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e54131510 0x555e54156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4aaaffd640 0 client.15297 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 --2- 172.21.15.107:0/3103781720 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e54130bf0 0x555e54155c50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.462+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 >> 172.21.15.107:0/3103781720 conn(0x555e5412ea20 msgr2=0x555e54138bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 shutdown_connections 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad448d740 1 -- 172.21.15.107:0/3103781720 wait complete. 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad448d740 20 client.15297 trim_cache size 0 max 16384 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 1, 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24562, 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:48.472 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 100.506761154, 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 127.28893542 2024-08-20T06:36:48.473 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 msgr2=0x7f4ab80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 0x7f4ab80b6090 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4ac40a4720 tx=0x7f4ac40d9db0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 msgr2=0x7f4acc19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ac400b970 tx=0x7f4ac400bd70 comp rx=0 tx=0).stop 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 shutdown_connections 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4acc108e60 0x7f4acc19e780 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4acc1084e0 0x7f4acc1993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4ab80b3bd0 0x7f4ab80b6090 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 --2- 172.21.15.107:0/883858174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4acc102ad0 0x7f4acc198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:48.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 >> 172.21.15.107:0/883858174 conn(0x7f4acc0fe540 msgr2=0x7f4acc100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:48.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 shutdown_connections 2024-08-20T06:36:48.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:48.463+0000 7f4ad2094640 1 -- 172.21.15.107:0/883858174 wait complete. 2024-08-20T06:36:49.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T06:36:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:49 smithi183 ceph-mon[22204]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T06:36:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:49 smithi193 ceph-mon[21944]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T06:36:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:49 smithi107 ceph-mon[18495]: pgmap v326: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T06:36:49.612 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 -- 172.21.15.107:0/3183035780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 msgr2=0x7fea9c101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 --2- 172.21.15.107:0/3183035780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c101db0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fea90009900 tx=0x7fea9002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 -- 172.21.15.107:0/3183035780 shutdown_connections 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 --2- 172.21.15.107:0/3183035780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c107d60 0x7fea9c10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 --2- 172.21.15.107:0/3183035780 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c1073e0 0x7fea9c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 --2- 172.21.15.107:0/3183035780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c101db0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 -- 172.21.15.107:0/3183035780 >> 172.21.15.107:0/3183035780 conn(0x7fea9c0fd440 msgr2=0x7fea9c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 -- 172.21.15.107:0/3183035780 shutdown_connections 2024-08-20T06:36:50.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.031+0000 7feaa2a86640 1 -- 172.21.15.107:0/3183035780 wait complete. 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 Processor -- start 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 -- start start 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c1073e0 0x7fea9c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c107d60 0x7fea9c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c111910 con 0x7fea9c1019d0 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c111790 con 0x7fea9c107d60 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa2a86640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c111a90 con 0x7fea9c1073e0 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36130/0 (socket says 172.21.15.107:36130) 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 -- 172.21.15.107:0/2228991477 learned_addr learned my addr 172.21.15.107:0/2228991477 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:50.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c1073e0 0x7fea9c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 -- 172.21.15.107:0/2228991477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c1073e0 msgr2=0x7fea9c19a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c1073e0 0x7fea9c19a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 -- 172.21.15.107:0/2228991477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c107d60 msgr2=0x7fea9c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa1023640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c107d60 0x7fea9c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c107d60 0x7fea9c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.032+0000 7feaa0822640 1 -- 172.21.15.107:0/2228991477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1a52c0 con 0x7fea9c1019d0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7feaa0822640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fea90009a20 tx=0x7fea90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7fea99ffb640 1 -- 172.21.15.107:0/2228991477 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9003f030 con 0x7fea9c1019d0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7fea99ffb640 1 -- 172.21.15.107:0/2228991477 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea900035f0 con 0x7fea9c1019d0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1a5550 con 0x7fea9c1019d0 2024-08-20T06:36:50.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7fea99ffb640 1 -- 172.21.15.107:0/2228991477 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90035070 con 0x7fea9c1019d0 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.033+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c1a5a80 con 0x7fea9c1019d0 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.035+0000 7feaa4e7f740 1 Processor -- start 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.035+0000 7feaa4e7f740 1 -- start start 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 0x555f13ddffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13debf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f13d566a0 con 0x555f13dec4a0 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f13d45ba0 con 0x555f13ddfbf0 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f13d49be0 con 0x555f13de0510 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea99ffb640 1 -- 172.21.15.107:0/2228991477 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fea90003aa0 con 0x7fea9c1019d0 2024-08-20T06:36:50.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36136/0 (socket says 172.21.15.107:36136) 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 -- 172.21.15.107:0/2353079832 learned_addr learned my addr 172.21.15.107:0/2353079832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa1023640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 0x555f13ddffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 -- 172.21.15.107:0/2353079832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 msgr2=0x555f13debf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa0822640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13debf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13debf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 -- 172.21.15.107:0/2353079832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 msgr2=0x555f13ddffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 0x555f13ddffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 -- 172.21.15.107:0/2353079832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13aa5540 con 0x555f13dec4a0 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7feaa1023640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 0x555f13ddffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:50.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fea88009a70 tx=0x7fea88009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad574e9dca1fe9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.036+0000 7fea86ffd640 1 -- 172.21.15.107:0/2353079832 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88017070 con 0x555f13dec4a0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7fea99ffb640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 0x7fea8c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7fea86ffd640 1 -- 172.21.15.107:0/2353079832 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea88013470 con 0x555f13dec4a0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7fea86ffd640 1 -- 172.21.15.107:0/2353079832 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88012630 con 0x555f13dec4a0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa1023640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 0x7fea8c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7fea99ffb640 1 -- 172.21.15.107:0/2228991477 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fea901044f0 con 0x7fea9c1019d0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2353079832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 msgr2=0x555f13dee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa1023640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 0x7fea8c0b6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea9c19aec0 tx=0x7fea94007530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fea88009a70 tx=0x7fea88009f30 comp rx=0 tx=0).stop 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2353079832 shutdown_connections 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13dec4a0 0x555f13dee890 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13debf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2353079832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13ddfbf0 0x555f13ddffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2353079832 >> 172.21.15.107:0/2353079832 conn(0x555f13ddda20 msgr2=0x555f13ddebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.037+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2353079832 shutdown_connections 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2353079832 wait complete. 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 -- start start 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 Processor -- start 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f13ba85f0 con 0x555f13ddfbf0 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f13ccc6c0 con 0x555f13dec4a0 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f13def150 con 0x555f13de0510 2024-08-20T06:36:50.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa4e7f740 10 client.0 _ll_register_callbacks cb 0x555f13d091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36138/0 (socket says 172.21.15.107:36138) 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.038+0000 7feaa0822640 1 -- 172.21.15.107:0/2866403436 learned_addr learned my addr 172.21.15.107:0/2866403436 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa1023640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 -- 172.21.15.107:0/2866403436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 msgr2=0x555f13de9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 -- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 msgr2=0x555f13de9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 -- 172.21.15.107:0/2866403436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13df84e0 con 0x555f13ddfbf0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa0822640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea900cb140 tx=0x7fea900cae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7991e4d94c493851 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90049040 con 0x555f13ddfbf0 2024-08-20T06:36:50.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fea900c76d0 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea88013470 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea900cb440 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa4e7f740 10 client.15309 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f13dfae70 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.039+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea88013470 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fea900d4060 con 0x555f13ddfbf0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 10 client.15309 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 10 client.15309 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 20 client.15309 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13dfd520 0x555f13dfe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 4 client.15309 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f13d585f0 con 0x555f13dfd520 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13dfd520 0x555f13dfe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7fea7bfff640 10 client.15309 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.040+0000 7fea9bfff640 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13dfd520 0x555f13dfe990 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29e63b32ed135be server_cookie=a26254b7c1c9174c in_seq=0 out_seq=0 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7fea7bfff640 1 -- 172.21.15.107:0/2866403436 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f13d585f0 con 0x555f13dfd520 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7fea7bfff640 10 client.15309 handle_command_reply: tid=0 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 client.15309 shutdown 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 msgr2=0x555f13de93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fea900cb140 tx=0x7fea900cae50 comp rx=0 tx=0).stop 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 shutdown_connections 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de9e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13dfd520 0x555f13dfe990 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/2866403436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de93d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 >> 172.21.15.107:0/2866403436 conn(0x555f13ddda20 msgr2=0x555f13de35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7fea7bfff640 0 client.15309 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:50.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 shutdown_connections 2024-08-20T06:36:50.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 1 -- 172.21.15.107:0/2866403436 wait complete. 2024-08-20T06:36:50.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.041+0000 7feaa4e7f740 20 client.15309 trim_cache size 0 max 16384 2024-08-20T06:36:50.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 Processor -- start 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 -- start start 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfccd0 con 0x555f13ddfbf0 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfd3a0 con 0x555f13dec4a0 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfda70 con 0x555f13de0510 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36142/0 (socket says 172.21.15.107:36142) 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa0822640 1 -- 172.21.15.107:0/1102223371 learned_addr learned my addr 172.21.15.107:0/1102223371 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa0822640 1 -- 172.21.15.107:0/1102223371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 msgr2=0x555f13de7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7fea9bfff640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.064+0000 7feaa1023640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa0822640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa0822640 1 -- 172.21.15.107:0/1102223371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 msgr2=0x555f13de8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa0822640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa0822640 1 -- 172.21.15.107:0/1102223371 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f13ddd850 con 0x555f13ddfbf0 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7fea9bfff640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:50.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa1023640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa0822640 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a450 tx=0x7fea90005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea4908fd18c3d2e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7fea867fc640 1 -- 172.21.15.107:0/1102223371 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90049070 con 0x555f13ddfbf0 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7fea867fc640 1 -- 172.21.15.107:0/1102223371 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea900cfa40 con 0x555f13ddfbf0 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7fea867fc640 1 -- 172.21.15.107:0/1102223371 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea900d4060 con 0x555f13ddfbf0 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 -- 172.21.15.107:0/1102223371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 msgr2=0x555f13de3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fea9002a450 tx=0x7fea90005e00 comp rx=0 tx=0).stop 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 -- 172.21.15.107:0/1102223371 shutdown_connections 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13de8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13de7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.065+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/1102223371 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13de3cf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- 172.21.15.107:0/1102223371 >> 172.21.15.107:0/1102223371 conn(0x555f13ddda20 msgr2=0x555f13dfc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- 172.21.15.107:0/1102223371 shutdown_connections 2024-08-20T06:36:50.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- 172.21.15.107:0/1102223371 wait complete. 2024-08-20T06:36:50.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- start start 2024-08-20T06:36:50.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 Processor -- start 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13e002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13e05210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfe810 con 0x555f13ddfbf0 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfe140 con 0x555f13dec4a0 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555f13dfeee0 con 0x555f13de0510 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa4e7f740 10 client.0 _ll_register_callbacks cb 0x7fea900c7370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.066+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36154/0 (socket says 172.21.15.107:36154) 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 -- 172.21.15.107:0/4003210166 learned_addr learned my addr 172.21.15.107:0/4003210166 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:50.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea9bfff640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13e002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa1023640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13e05210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 -- 172.21.15.107:0/4003210166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 msgr2=0x555f13e002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13e002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 -- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 msgr2=0x555f13e05210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13e05210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 -- 172.21.15.107:0/4003210166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f13dde4a0 con 0x555f13ddfbf0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa1023640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13e05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa0822640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fea90032fd0 tx=0x7fea900c7970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c9c5afba7c9829 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90049070 con 0x555f13ddfbf0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fea900c5b60 con 0x555f13ddfbf0 2024-08-20T06:36:50.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea900cfa40 con 0x555f13ddfbf0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fea900ac050 con 0x555f13ddfbf0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa4e7f740 10 client.15315 fetch_fsmap learned FSMap version 24 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x555f13dde4a0 con 0x555f13ddfbf0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.067+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fea900cfa40 con 0x555f13ddfbf0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fea90104ae0 con 0x555f13ddfbf0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 10 client.15315 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 20 client.15315 populate_metadata read hostname 'smithi107' 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13e6b260 0x555f13e69f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 4 client.15315 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "status"}) -- 0x555f13d54090 con 0x555f13e6b260 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea9bfff640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13e6b260 0x555f13e69f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:50.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea7bfff640 10 client.15315 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea9bfff640 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13e6b260 0x555f13e69f70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4378ee40e2e12bf3 server_cookie=13031059104099e1 in_seq=0 out_seq=0 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea7bfff640 1 -- 172.21.15.107:0/4003210166 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555f13d54090 con 0x555f13e6b260 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7fea7bfff640 10 client.15315 handle_command_reply: tid=0 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.068+0000 7feaa4e7f740 1 client.15315 shutdown 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 msgr2=0x555f13dffd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fea90032fd0 tx=0x7fea900c7970 comp rx=0 tx=0).stop 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 shutdown_connections 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555f13dec4a0 0x555f13e05210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555f13e6b260 0x555f13e69f70 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7fea7bfff640 0 client.15315 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555f13de0510 0x555f13e002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 --2- 172.21.15.107:0/4003210166 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555f13ddfbf0 0x555f13dffd70 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 >> 172.21.15.107:0/4003210166 conn(0x555f13ddda20 msgr2=0x555f13ddea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 shutdown_connections 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 1 -- 172.21.15.107:0/4003210166 wait complete. 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa4e7f740 20 client.15315 trim_cache size 0 max 16384 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stdout: "cluster_fsid": "1c223f02-5ebd-11ef-bccf-c7b262605968", 2024-08-20T06:36:50.077 INFO:teuthology.orchestra.run.smithi107.stdout: "whoami": 0, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 24572, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "want_state": "up:active", 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "state": "up:active", 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs", 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "rank_uptime": 109.153552484, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "mdsmap_epoch": 24, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch": 73, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_epoch_barrier": 71, 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout: "uptime": 119.766640643 2024-08-20T06:36:50.078 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 msgr2=0x7fea8c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 0x7fea8c0b6110 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea9c19aec0 tx=0x7fea94007530 comp rx=0 tx=0).stop 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 msgr2=0x7fea9c199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fea90009a20 tx=0x7fea90004270 comp rx=0 tx=0).stop 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 shutdown_connections 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fea9c107d60 0x7fea9c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fea8c0b3c50 0x7fea8c0b6110 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea9c1073e0 0x7fea9c19a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 --2- 172.21.15.107:0/2228991477 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea9c1019d0 0x7fea9c199ee0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:50.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.069+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 >> 172.21.15.107:0/2228991477 conn(0x7fea9c0fd440 msgr2=0x7fea9c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:50.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.070+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 shutdown_connections 2024-08-20T06:36:50.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:50.070+0000 7feaa2a86640 1 -- 172.21.15.107:0/2228991477 wait complete. 2024-08-20T06:36:50.823 INFO:teuthology.run_tasks:Running task sequential... 2024-08-20T06:36:50.834 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-20T06:36:50.834 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-20T06:36:50.834 INFO:teuthology.task.print:Enabling mgr modules 2024-08-20T06:36:50.834 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-08-20T06:36:50.897 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-08-20T06:36:50.897 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-08-20T06:36:50.897 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-08-20T06:36:50.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-20T06:36:50.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:50.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:50.898 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-08-20T06:36:50.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:50.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-08-20T06:36:50.981 INFO:teuthology.orchestra.run.smithi107.stderr:Cannot find device "ceph-brx" 2024-08-20T06:36:50.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T06:36:50.982 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-08-20T06:36:50.982 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-08-20T06:36:50.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-20T06:36:51.267 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:51 smithi107 ceph-mon[18495]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:51 smithi183 ceph-mon[22204]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:51 smithi193 ceph-mon[21944]: pgmap v327: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 -- 172.21.15.107:0/1988297613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6404107d20 msgr2=0x7f640410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 --2- 172.21.15.107:0/1988297613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6404107d20 0x7f640410e520 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009ca0 tx=0x7f63fc02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 -- 172.21.15.107:0/1988297613 shutdown_connections 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 --2- 172.21.15.107:0/1988297613 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6404107d20 0x7f640410e520 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 --2- 172.21.15.107:0/1988297613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f64041073a0 0x7f64041077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 --2- 172.21.15.107:0/1988297613 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 0x7f6404101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 -- 172.21.15.107:0/1988297613 >> 172.21.15.107:0/1988297613 conn(0x7f64040fd400 msgr2=0x7f64040ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.703+0000 7f6409828640 1 -- 172.21.15.107:0/1988297613 shutdown_connections 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.704+0000 7f6409828640 1 -- 172.21.15.107:0/1988297613 wait complete. 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.704+0000 7f6409828640 1 Processor -- start 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.704+0000 7f6409828640 1 -- start start 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 0x7f6404199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404107d20 0x7f640419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f64041118e0 con 0x7f64041073a0 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111760 con 0x7f6404101990 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111a60 con 0x7f6404107d20 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:51.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36172/0 (socket says 172.21.15.107:36172) 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 -- 172.21.15.107:0/2531500854 learned_addr learned my addr 172.21.15.107:0/2531500854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6402ffd640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 0x7f6404199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f64037fe640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404107d20 0x7f640419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 -- 172.21.15.107:0/2531500854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404107d20 msgr2=0x7f640419a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404107d20 0x7f640419a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 -- 172.21.15.107:0/2531500854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 msgr2=0x7f6404199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 0x7f6404199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 -- 172.21.15.107:0/2531500854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64041a50b0 con 0x7f64041073a0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f63fa7fc640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b4a0 tx=0x7f63ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec014070 con 0x7f64041073a0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64041a53a0 con 0x7f64041073a0 2024-08-20T06:36:51.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63ec010aa0 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec017b70 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.705+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64041a58e0 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.707+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6404101d70 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.708+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f63ec017070 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.709+0000 7f6408826640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 0x7f63f00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.709+0000 7f6402ffd640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 0x7f63f00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.709+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63ec0e33d0 con 0x7f64041073a0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.709+0000 7f6402ffd640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 0x7f63f00b6090 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63f400bbe0 tx=0x7f63f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:51.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.712+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f63ec0aaa20 con 0x7f64041073a0 2024-08-20T06:36:51.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.925+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f640402e340 con 0x7f64041073a0 2024-08-20T06:36:51.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.926+0000 7f6408826640 1 -- 172.21.15.107:0/2531500854 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f63ec0aa320 con 0x7f64041073a0 2024-08-20T06:36:51.927 INFO:teuthology.orchestra.run.smithi107.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-08-20T06:36:51.928 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.927+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 msgr2=0x7f63f00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.927+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 0x7f63f00b6090 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f63f400bbe0 tx=0x7f63f400b390 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.927+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 msgr2=0x7f640419a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00b4a0 tx=0x7f63ec00b960 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 shutdown_connections 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63f00b3bd0 0x7f63f00b6090 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6404107d20 0x7f640419a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f64041073a0 0x7f640419a1f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 --2- 172.21.15.107:0/2531500854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6404101990 0x7f6404199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 >> 172.21.15.107:0/2531500854 conn(0x7f64040fd400 msgr2=0x7f64040ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 shutdown_connections 2024-08-20T06:36:51.929 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:51.928+0000 7f6409828640 1 -- 172.21.15.107:0/2531500854 wait complete. 2024-08-20T06:36:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:52 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2531500854' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T06:36:52.564 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2531500854' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi107.front.sepia.ceph.com 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:52.591 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-20T06:36:52.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-20T06:36:52.592 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-20T06:36:52.592 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-20T06:36:52.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:52.592 DEBUG:teuthology.orchestra.run.smithi107:> ip addr 2024-08-20T06:36:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: inet 127.0.0.1/8 scope host lo 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 ::1/128 scope host 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0a brd ff:ff:ff:ff:ff:ff 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f0 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:d8:a2:0b brd ff:ff:ff:ff:ff:ff 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: altname enp7s0f1 2024-08-20T06:36:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:ea brd ff:ff:ff:ff:ff:ff 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: link/ether 0c:c4:7a:8f:47:eb brd ff:ff:ff:ff:ff:ff 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: inet 172.21.15.107/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-08-20T06:36:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: valid_lft forever preferred_lft forever 2024-08-20T06:36:52.610 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add name ceph-brx type bridge 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr flush dev ceph-brx 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set ceph-brx up 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-20T06:36:52.611 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-08-20T06:36:52.696 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T06:36:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:52 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2531500854' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T06:36:52.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:52.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:52.813 DEBUG:teuthology.orchestra.run.smithi107:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-20T06:36:52.838 INFO:teuthology.orchestra.run.smithi107.stdout:1 2024-08-20T06:36:52.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:52.840 DEBUG:teuthology.orchestra.run.smithi107:> route 2024-08-20T06:36:52.914 INFO:teuthology.orchestra.run.smithi107.stdout:Kernel IP routing table 2024-08-20T06:36:52.914 INFO:teuthology.orchestra.run.smithi107.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-20T06:36:52.914 INFO:teuthology.orchestra.run.smithi107.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-20T06:36:52.914 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-20T06:36:52.914 INFO:teuthology.orchestra.run.smithi107.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-20T06:36:52.915 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-08-20T06:36:53.002 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T06:36:53.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:53.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:53.202 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-08-20T06:36:53.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:53.218 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list-id 2024-08-20T06:36:53.274 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T06:36:53.274 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-08-20T06:36:53.274 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:53.274 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-20T06:36:53.274 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-08-20T06:36:53.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T06:36:53.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:53 smithi107 ceph-mon[18495]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:53.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:53.396 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-20T06:36:53.396 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-08-20T06:36:53.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T06:36:53.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:53.531 DEBUG:teuthology.orchestra.run.smithi107:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T06:36:53.532 DEBUG:teuthology.orchestra.run.smithi107:> set -e 2024-08-20T06:36:53.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set brx.0 up 2024-08-20T06:36:53.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link set dev brx.0 master ceph-brx 2024-08-20T06:36:53.532 DEBUG:teuthology.orchestra.run.smithi107:> ') 2024-08-20T06:36:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:53 smithi193 ceph-mon[21944]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:53.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T06:36:53.602 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:53.608 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-08-20T06:36:53.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T06:36:53.609 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:53.624 INFO:teuthology.orchestra.run.smithi107.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-20T06:36:53.624 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T06:36:53.624 DEBUG:teuthology.orchestra.run.smithi107:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:53.680 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-08-20T06:36:53.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:53 smithi183 ceph-mon[22204]: pgmap v328: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:36:53.747 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T06:36:53.803 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-08-20T06:36:53.804 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T06:36:53.805 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/92e230458d276e275ac65b94097b4a19dee43001f994a8fefd1bbec7e6cfe13a/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/809e40233fde0d7379bcf677318372a62badd70b8c17ec1666ad0aca6d215000/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c1f6401a578578aa45864874a4a5cb150f3e2c98632c091aee38b8d9aeca891b/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6d70c48ac21ef0789ccf2b859ce393b9461ef36d1d2f7864cd9e8e7ebd8593b2/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/17417e06e2b29d95bbdd4023c0c5ec9e85bba00ef195657fca407b2be482b68e/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/90436425c5b4851fe2108cd3746e94dff4a1c165fbacc0a4c4ea8736efc49877/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4f70b883176ae715f54e5b79720d06790167c68f7b89c9b41db3c4edcb865e21/merged 2024-08-20T06:36:53.806 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6ef927b76dac4881ded7ef556304724a77e497707bd724a4e0aa4e8f72659038/merged 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/722d75d384170afe0ed952d905a1ddf93e2c02ae11d8d81e0a00cc2198326784/merged 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5e964009e42c7e20217bf7a2f389b37cf2101ce44689a150de8de0b13a269303/merged 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:53.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:53.807 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-08-20T06:36:53.859 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-20T06:36:53.859 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/6bb58bf9-4f8e-4e40-be28-10b8c824fb5d --client_fs=cephfs) 2024-08-20T06:36:53.902 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-08-20T06:36:53.927 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T06:36:53.984 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-08-20T06:36:53.984 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-08-20T06:36:53.984 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-08-20T06:36:53.984 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-08-20T06:36:53.984 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-08-20T06:36:53.985 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/92e230458d276e275ac65b94097b4a19dee43001f994a8fefd1bbec7e6cfe13a/merged 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/809e40233fde0d7379bcf677318372a62badd70b8c17ec1666ad0aca6d215000/merged 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-08-20T06:36:53.986 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c1f6401a578578aa45864874a4a5cb150f3e2c98632c091aee38b8d9aeca891b/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6d70c48ac21ef0789ccf2b859ce393b9461ef36d1d2f7864cd9e8e7ebd8593b2/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/17417e06e2b29d95bbdd4023c0c5ec9e85bba00ef195657fca407b2be482b68e/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/90436425c5b4851fe2108cd3746e94dff4a1c165fbacc0a4c4ea8736efc49877/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4f70b883176ae715f54e5b79720d06790167c68f7b89c9b41db3c4edcb865e21/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6ef927b76dac4881ded7ef556304724a77e497707bd724a4e0aa4e8f72659038/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/722d75d384170afe0ed952d905a1ddf93e2c02ae11d8d81e0a00cc2198326784/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5e964009e42c7e20217bf7a2f389b37cf2101ce44689a150de8de0b13a269303/merged 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:53.987 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:53.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:53.988 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-08-20T06:36:54.033 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:2024-08-20T06:36:54.033+0000 7fc38a3dc480 -1 init, newargv = 0x560eb83ec060 newargc=15 2024-08-20T06:36:54.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[70495]: starting ceph client 2024-08-20T06:36:54.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[70495]: starting fuse 2024-08-20T06:36:55.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo modprobe fuse 2024-08-20T06:36:55.173 DEBUG:teuthology.orchestra.run.smithi107:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/proc 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/sys 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/dev 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/security 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/shm 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/pts 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/run 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/cgroup 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/pstore 2024-08-20T06:36:55.231 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/bpf 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/ 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/selinux 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/hugepages 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/dev/mqueue 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/debug 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/tracing 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysusers.service 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/kernel/config 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/sys/fs/fuse/connections 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T06:36:55.232 INFO:teuthology.orchestra.run.smithi107.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/1000 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/ceph 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/92e230458d276e275ac65b94097b4a19dee43001f994a8fefd1bbec7e6cfe13a/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/809e40233fde0d7379bcf677318372a62badd70b8c17ec1666ad0aca6d215000/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/run/user/0 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/c1f6401a578578aa45864874a4a5cb150f3e2c98632c091aee38b8d9aeca891b/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6d70c48ac21ef0789ccf2b859ce393b9461ef36d1d2f7864cd9e8e7ebd8593b2/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/17417e06e2b29d95bbdd4023c0c5ec9e85bba00ef195657fca407b2be482b68e/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/90436425c5b4851fe2108cd3746e94dff4a1c165fbacc0a4c4ea8736efc49877/merged 2024-08-20T06:36:55.233 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/4f70b883176ae715f54e5b79720d06790167c68f7b89c9b41db3c4edcb865e21/merged 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/6ef927b76dac4881ded7ef556304724a77e497707bd724a4e0aa4e8f72659038/merged 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/722d75d384170afe0ed952d905a1ddf93e2c02ae11d8d81e0a00cc2198326784/merged 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/var/lib/containers/storage/overlay/5e964009e42c7e20217bf7a2f389b37cf2101ce44689a150de8de0b13a269303/merged 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:36:55.234 INFO:teuthology.orchestra.run.smithi107.stdout:/home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:55.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:55.235 DEBUG:teuthology.orchestra.run.smithi107:> ls /sys/fs/fuse/connections 2024-08-20T06:36:55.292 INFO:teuthology.orchestra.run.smithi107.stdout:121 2024-08-20T06:36:55.292 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo stdin-killer -- python3 -c ' 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> import glob 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> import re 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> import os 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> import subprocess 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> def _find_admin_socket(client_name): 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> files = glob.glob(asok_path) 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-08-20T06:36:55.293 DEBUG:teuthology.orchestra.run.smithi107:> 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> # Given a non-glob path, it better be there 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> if "*" not in asok_path: 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> assert(len(files) == 1) 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> return files[0] 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> for f in files: 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> contents = proc_f.read() 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> if mountpoint in contents: 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> return f 2024-08-20T06:36:55.294 DEBUG:teuthology.orchestra.run.smithi107:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-20T06:36:55.295 DEBUG:teuthology.orchestra.run.smithi107:> 2024-08-20T06:36:55.295 DEBUG:teuthology.orchestra.run.smithi107:> print(_find_admin_socket("client.0")) 2024-08-20T06:36:55.295 DEBUG:teuthology.orchestra.run.smithi107:> ' 2024-08-20T06:36:55.413 INFO:teuthology.orchestra.run.smithi107.stdout:/var/run/ceph/ceph-client.0.70495.asok 2024-08-20T06:36:55.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T06:36:55.422 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:55 smithi107 ceph-mon[18495]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:36:55.423 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.70495.asok 2024-08-20T06:36:55.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:55.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.70495.asok status 2024-08-20T06:36:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:55 smithi193 ceph-mon[21944]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:36:55.557 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:36:55.557 INFO:teuthology.orchestra.run.smithi107.stdout: "metadata": { 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_sha1": "ef4275410d8483698016fe484b2a3c664080b4c1", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "entity_id": "0", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": "smithi107", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "pid": "70495", 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "root": "/volumes/qa/sv_0/6bb58bf9-4f8e-4e40-be28-10b8c824fb5d" 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_count": 0, 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "dentry_pinned_count": 0, 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "id": 15327, 2024-08-20T06:36:55.558 INFO:teuthology.orchestra.run.smithi107.stdout: "inst": { 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": { 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "client", 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "num": 15327 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2858007165 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": { 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "192.168.0.1:0", 2024-08-20T06:36:55.559 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 2858007165 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "inst_str": "client.15327 192.168.0.1:0/2858007165", 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "addr_str": "192.168.0.1:0/2858007165", 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "inode_count": 5, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_epoch": 24, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch": 73, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_epoch_barrier": 0, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "blocklisted": false, 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout: "fs_name": "cephfs" 2024-08-20T06:36:55.560 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:36:55.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:55.566 DEBUG:teuthology.orchestra.run.smithi107:> stat --file-system '--printf=%T 2024-08-20T06:36:55.566 DEBUG:teuthology.orchestra.run.smithi107:> ' -- /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:55.583 INFO:teuthology.orchestra.run.smithi107.stdout:fuseblk 2024-08-20T06:36:55.583 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:55.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:36:55.583 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-20T06:36:55.651 INFO:teuthology.run_tasks:Running task exec... 2024-08-20T06:36:55.662 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T06:36:55.663 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi107.front.sepia.ceph.com 2024-08-20T06:36:55.663 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-08-20T06:36:55.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:55 smithi183 ceph-mon[22204]: pgmap v329: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:36:55.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 -- 172.21.15.107:0/1000276893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94101580 msgr2=0x7fee94101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:55.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/1000276893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94101580 0x7fee94101980 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fee90009930 tx=0x7fee9002a420 comp rx=0 tx=0).stop 2024-08-20T06:36:55.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 -- 172.21.15.107:0/1000276893 shutdown_connections 2024-08-20T06:36:55.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/1000276893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94103090 0x7fee94109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/1000276893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 0x7fee94102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/1000276893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94101580 0x7fee94101980 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 -- 172.21.15.107:0/1000276893 >> 172.21.15.107:0/1000276893 conn(0x7fee940fce60 msgr2=0x7fee940ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 -- 172.21.15.107:0/1000276893 shutdown_connections 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.790+0000 7fee9c5a9640 1 -- 172.21.15.107:0/1000276893 wait complete. 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 Processor -- start 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 -- start start 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 0x7fee94198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 0x7fee941990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:55.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fee9410e230 con 0x7fee94103090 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fee9410e0b0 con 0x7fee941026d0 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9c5a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fee9410e3b0 con 0x7fee94101580 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60104/0 (socket says 172.21.15.107:60104) 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 -- 172.21.15.107:0/3715736242 learned_addr learned my addr 172.21.15.107:0/3715736242 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee99b44640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 0x7fee941990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9a345640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 0x7fee94198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 -- 172.21.15.107:0/3715736242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 msgr2=0x7fee94198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 0x7fee94198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 -- 172.21.15.107:0/3715736242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 msgr2=0x7fee941990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 0x7fee941990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 -- 172.21.15.107:0/3715736242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9419a590 con 0x7fee94103090 2024-08-20T06:36:55.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9a345640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 0x7fee94198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee9ab46640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00eaa0 tx=0x7fee8c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.791+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00cdb0 con 0x7fee94103090 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.792+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9419a880 con 0x7fee94103090 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.792+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fee8c005020 con 0x7fee94103090 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.792+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8c0116d0 con 0x7fee94103090 2024-08-20T06:36:55.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.792+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee941a0f50 con 0x7fee94103090 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.792+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee941089e0 con 0x7fee94103090 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.794+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fee8c004a40 con 0x7fee94103090 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.794+0000 7fee8b7fe640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 0x7fee800b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.795+0000 7fee9a345640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 0x7fee800b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.795+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fee8c0e4160 con 0x7fee94103090 2024-08-20T06:36:55.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.795+0000 7fee9a345640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 0x7fee800b6090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee900057c0 tx=0x7fee90005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:55.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:55.797+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fee8c016030 con 0x7fee94103090 2024-08-20T06:36:56.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.008+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fee9402e520 con 0x7fee94103090 2024-08-20T06:36:56.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.245+0000 7fee8b7fe640 1 -- 172.21.15.107:0/3715736242 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7fee8c0ab7b0 con 0x7fee94103090 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 msgr2=0x7fee800b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 0x7fee800b6090 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fee900057c0 tx=0x7fee90005fd0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 msgr2=0x7fee94199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00eaa0 tx=0x7fee8c00ef60 comp rx=0 tx=0).stop 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 shutdown_connections 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fee94103090 0x7fee94199cd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fee941026d0 0x7fee941990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fee800b3bd0 0x7fee800b6090 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 --2- 172.21.15.107:0/3715736242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fee94101580 0x7fee94198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 >> 172.21.15.107:0/3715736242 conn(0x7fee940fce60 msgr2=0x7fee94105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:56.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 shutdown_connections 2024-08-20T06:36:56.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.249+0000 7fee9c5a9640 1 -- 172.21.15.107:0/3715736242 wait complete. 2024-08-20T06:36:56.261 DEBUG:teuthology.orchestra.run.smithi107:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-08-20T06:36:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 -- 172.21.15.107:0/1238029993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 msgr2=0x7fa680101fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 --2- 172.21.15.107:0/1238029993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa680101fc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa670009900 tx=0x7fa67002a410 comp rx=0 tx=0).stop 2024-08-20T06:36:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 -- 172.21.15.107:0/1238029993 shutdown_connections 2024-08-20T06:36:56.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 --2- 172.21.15.107:0/1238029993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 0x7fa680109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 --2- 172.21.15.107:0/1238029993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa680102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 --2- 172.21.15.107:0/1238029993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa680101fc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 -- 172.21.15.107:0/1238029993 >> 172.21.15.107:0/1238029993 conn(0x7fa6800fc5b0 msgr2=0x7fa6800fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 -- 172.21.15.107:0/1238029993 shutdown_connections 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 -- 172.21.15.107:0/1238029993 wait complete. 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.364+0000 7fa685ba9640 1 Processor -- start 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 -- start start 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa68019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 0x7fa68019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010d100 con 0x7fa680101bc0 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010cf80 con 0x7fa680102500 2024-08-20T06:36:56.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa685ba9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa68010d280 con 0x7fa680103070 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60126/0 (socket says 172.21.15.107:60126) 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 -- 172.21.15.107:0/3180560228 learned_addr learned my addr 172.21.15.107:0/3180560228 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67effd640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa68019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67ffff640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 0x7fa68019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 -- 172.21.15.107:0/3180560228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 msgr2=0x7fa68019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 0x7fa68019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 -- 172.21.15.107:0/3180560228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 msgr2=0x7fa68019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa68019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 -- 172.21.15.107:0/3180560228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6801a0570 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67effd640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa68019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67f7fe640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa670009230 tx=0x7fa6700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.365+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67003f030 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.366+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa670003c90 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.366+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6801a0800 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.366+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa670035070 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.366+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a0dc0 con 0x7fa680101bc0 2024-08-20T06:36:56.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.366+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa680108a90 con 0x7fa680101bc0 2024-08-20T06:36:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.368+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fa670042030 con 0x7fa680101bc0 2024-08-20T06:36:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.369+0000 7fa67cff9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 0x7fa6680b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.369+0000 7fa67effd640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 0x7fa6680b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.369+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa670104820 con 0x7fa680101bc0 2024-08-20T06:36:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.369+0000 7fa67effd640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 0x7fa6680b6110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa674009930 tx=0x7fa674008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:56.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.371+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa6700cbe70 con 0x7fa680101bc0 2024-08-20T06:36:56.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:56 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T06:36:56.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.573+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fa680063420 con 0x7fa6680b3c50 2024-08-20T06:36:56.580 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T06:36:56.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.593+0000 7fa67cff9640 1 -- 172.21.15.107:0/3180560228 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fa680063420 con 0x7fa6680b3c50 2024-08-20T06:36:56.594 INFO:teuthology.orchestra.run.smithi107.stdout:{} 2024-08-20T06:36:56.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 msgr2=0x7fa6680b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 0x7fa6680b6110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa674009930 tx=0x7fa674008040 comp rx=0 tx=0).stop 2024-08-20T06:36:56.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 msgr2=0x7fa6801011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa670009230 tx=0x7fa6700042f0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 shutdown_connections 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa6680b3c50 0x7fa6680b6110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa680103070 0x7fa68019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa680102500 0x7fa68019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 --2- 172.21.15.107:0/3180560228 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa680101bc0 0x7fa6801011f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 >> 172.21.15.107:0/3180560228 conn(0x7fa6800fc5b0 msgr2=0x7fa680105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 shutdown_connections 2024-08-20T06:36:56.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.595+0000 7fa685ba9640 1 -- 172.21.15.107:0/3180560228 wait complete. 2024-08-20T06:36:56.606 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-20T06:36:56.618 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-08-20T06:36:56.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-20T06:36:56.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:56 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} : dispatch 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- 172.21.15.107:0/1267694200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 msgr2=0x7f270c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- 172.21.15.107:0/1267694200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c109a20 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2704009cd0 tx=0x7f270402a8e0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- 172.21.15.107:0/1267694200 shutdown_connections 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- 172.21.15.107:0/1267694200 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c109a20 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- 172.21.15.107:0/1267694200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c1026e0 0x7f270c102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- 172.21.15.107:0/1267694200 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c101590 0x7f270c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- 172.21.15.107:0/1267694200 >> 172.21.15.107:0/1267694200 conn(0x7f270c0fce70 msgr2=0x7f270c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- 172.21.15.107:0/1267694200 shutdown_connections 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- 172.21.15.107:0/1267694200 wait complete. 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 Processor -- start 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- start start 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c101590 0x7f270c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 0x7f270c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f270c10d0b0 con 0x7f270c1030a0 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f270c10cf30 con 0x7f270c1026e0 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.778+0000 7f2713523640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f270c10d230 con 0x7f270c101590 2024-08-20T06:36:56.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60146/0 (socket says 172.21.15.107:60146) 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 -- 172.21.15.107:0/1460653028 learned_addr learned my addr 172.21.15.107:0/1460653028 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f27112bf640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c101590 0x7f270c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2710abe640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 0x7f270c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 -- 172.21.15.107:0/1460653028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c101590 msgr2=0x7f270c06a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c101590 0x7f270c06a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 -- 172.21.15.107:0/1460653028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 msgr2=0x7f270c19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 0x7f270c19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 -- 172.21.15.107:0/1460653028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f270c1a04f0 con 0x7f270c1030a0 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2710abe640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 0x7f270c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2711ac0640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f27040097b0 tx=0x7f2704003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:56.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704004450 con 0x7f270c1030a0 2024-08-20T06:36:56.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f270c1a0780 con 0x7f270c1030a0 2024-08-20T06:36:56.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2704003e10 con 0x7f270c1030a0 2024-08-20T06:36:56.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2704046480 con 0x7f270c1030a0 2024-08-20T06:36:56.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.779+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f270c1a0cd0 con 0x7f270c1030a0 2024-08-20T06:36:56.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.780+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f270c108900 con 0x7f270c1030a0 2024-08-20T06:36:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.782+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f27040305a0 con 0x7f270c1030a0 2024-08-20T06:36:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.782+0000 7f27027fc640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 0x7f26f80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.782+0000 7f27112bf640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 0x7f26f80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.782+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f270402e030 con 0x7f270c1030a0 2024-08-20T06:36:56.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.783+0000 7f27112bf640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 0x7f26f80b6090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005d00 tx=0x7f26fc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:56.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.785+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f270404b060 con 0x7f270c1030a0 2024-08-20T06:36:56.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:36:56.578+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:36:56.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:36:56.578+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:36:56.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:36:56.578+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:36:56.861 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:36:56.578+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:36:56.862 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:36:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:36:56.578+0000 7f849290c640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T06:36:56.963 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.963+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f270c02e590 con 0x7f270c1030a0 2024-08-20T06:36:56.964 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.964+0000 7f27027fc640 1 -- 172.21.15.107:0/1460653028 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7f27040cce60 con 0x7f270c1030a0 2024-08-20T06:36:56.965 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:56.965 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:36:56.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 msgr2=0x7f26f80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 0x7f26f80b6090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f26fc005d00 tx=0x7f26fc00b410 comp rx=0 tx=0).stop 2024-08-20T06:36:56.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 msgr2=0x7f270c19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:56.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f27040097b0 tx=0x7f2704003c20 comp rx=0 tx=0).stop 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 shutdown_connections 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f270c1030a0 0x7f270c19fc30 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f270c1026e0 0x7f270c19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f26f80b3bd0 0x7f26f80b6090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 --2- 172.21.15.107:0/1460653028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f270c101590 0x7f270c06a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 >> 172.21.15.107:0/1460653028 conn(0x7f270c0fce70 msgr2=0x7f270c105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 shutdown_connections 2024-08-20T06:36:56.968 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:56.966+0000 7f2713523640 1 -- 172.21.15.107:0/1460653028 wait complete. 2024-08-20T06:36:56.978 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T06:30:39.511744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-20T06:34:56.266774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-20T06:34:58.530080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T06:36:56.979 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 -- 172.21.15.107:0/975069390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 msgr2=0x7f3278103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 --2- 172.21.15.107:0/975069390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278103c50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f326c009930 tx=0x7f326c02a420 comp rx=0 tx=0).stop 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 -- 172.21.15.107:0/975069390 shutdown_connections 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 --2- 172.21.15.107:0/975069390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 0x7f327810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 --2- 172.21.15.107:0/975069390 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278103c50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 --2- 172.21.15.107:0/975069390 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102680 0x7f3278102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 -- 172.21.15.107:0/975069390 >> 172.21.15.107:0/975069390 conn(0x7f32780fdf40 msgr2=0x7f3278100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 -- 172.21.15.107:0/975069390 shutdown_connections 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.080+0000 7f327d57b640 1 -- 172.21.15.107:0/975069390 wait complete. 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 Processor -- start 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 -- start start 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102680 0x7f3278076a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 0x7f32780757b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f327810e200 con 0x7f32781037d0 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f327810e080 con 0x7f3278102680 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f327d57b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f327810e380 con 0x7f3278104190 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60158/0 (socket says 172.21.15.107:60158) 2024-08-20T06:36:57.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 -- 172.21.15.107:0/605173705 learned_addr learned my addr 172.21.15.107:0/605173705 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32777fe640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 0x7f32780757b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 -- 172.21.15.107:0/605173705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 msgr2=0x7f32780757b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 0x7f32780757b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 -- 172.21.15.107:0/605173705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102680 msgr2=0x7f3278076a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102680 0x7f3278076a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.081+0000 7f32767fc640 1 -- 172.21.15.107:0/605173705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3278076020 con 0x7f32781037d0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f32767fc640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f326c005ab0 tx=0x7f326c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c03f030 con 0x7f32781037d0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32781a2a90 con 0x7f32781037d0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f326c003c90 con 0x7f32781037d0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c035070 con 0x7f32781037d0 2024-08-20T06:36:57.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.082+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32781a2fd0 con 0x7f32781037d0 2024-08-20T06:36:57.086 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.083+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3278102a80 con 0x7f32781037d0 2024-08-20T06:36:57.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.084+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f326c042030 con 0x7f32781037d0 2024-08-20T06:36:57.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.085+0000 7f325ffff640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.085+0000 7f3276ffd640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.085+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f326c104860 con 0x7f32781037d0 2024-08-20T06:36:57.087 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.085+0000 7f3276ffd640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3268007be0 tx=0x7f3268005f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.088+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f326c0cbeb0 con 0x7f32781037d0 2024-08-20T06:36:57.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.267+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f327802e370 con 0x7f32781037d0 2024-08-20T06:36:57.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.267+0000 7f325ffff640 1 -- 172.21.15.107:0/605173705 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f326c0d4180 con 0x7f32781037d0 2024-08-20T06:36:57.268 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-08-20T06:36:57.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 msgr2=0x7f32600b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3268007be0 tx=0x7f3268005f70 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 msgr2=0x7f3278075270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f326c005ab0 tx=0x7f326c0042f0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 shutdown_connections 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f32600b3bd0 0x7f32600b6090 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3278104190 0x7f32780757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f32781037d0 0x7f3278075270 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 --2- 172.21.15.107:0/605173705 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3278102680 0x7f3278076a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 >> 172.21.15.107:0/605173705 conn(0x7f32780fdf40 msgr2=0x7f3278106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 shutdown_connections 2024-08-20T06:36:57.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.269+0000 7f327d57b640 1 -- 172.21.15.107:0/605173705 wait complete. 2024-08-20T06:36:57.281 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-20T06:36:57.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 -- 172.21.15.107:0/2115929337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 msgr2=0x7fdfc0102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2115929337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0102a80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009930 tx=0x7fdfb002a420 comp rx=0 tx=0).stop 2024-08-20T06:36:57.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 -- 172.21.15.107:0/2115929337 shutdown_connections 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2115929337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc0104190 0x7fdfc010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2115929337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc01037d0 0x7fdfc0103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2115929337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0102a80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 -- 172.21.15.107:0/2115929337 >> 172.21.15.107:0/2115929337 conn(0x7fdfc00fdf40 msgr2=0x7fdfc0100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 -- 172.21.15.107:0/2115929337 shutdown_connections 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.382+0000 7fdfc5085640 1 -- 172.21.15.107:0/2115929337 wait complete. 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 Processor -- start 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 -- start start 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 0x7fdfc0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc0104190 0x7fdfc0199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010e1a0 con 0x7fdfc0102680 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010e020 con 0x7fdfc01037d0 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfc5085640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc010e320 con 0x7fdfc0104190 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfbed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfbed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60176/0 (socket says 172.21.15.107:60176) 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.383+0000 7fdfbed9d640 1 -- 172.21.15.107:0/2373356933 learned_addr learned my addr 172.21.15.107:0/2373356933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbf59e640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc0104190 0x7fdfc0199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbe59c640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 0x7fdfc0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 -- 172.21.15.107:0/2373356933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc0104190 msgr2=0x7fdfc0199c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc0104190 0x7fdfc0199c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 -- 172.21.15.107:0/2373356933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 msgr2=0x7fdfc0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 0x7fdfc0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 -- 172.21.15.107:0/2373356933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc019a4c0 con 0x7fdfc0102680 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbe59c640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 0x7fdfc0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfbed9d640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0005ab0 tx=0x7fdfb00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb003f030 con 0x7fdfc0102680 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdfb0003c90 con 0x7fdfc0102680 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc019a750 con 0x7fdfc0102680 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0035070 con 0x7fdfc0102680 2024-08-20T06:36:57.388 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.384+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc01a0e80 con 0x7fdfc0102680 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.385+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc0102a80 con 0x7fdfc0102680 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.386+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7fdfb0042030 con 0x7fdfc0102680 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.387+0000 7fdfa7fff640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 0x7fdfa80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.387+0000 7fdfbe59c640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 0x7fdfa80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.387+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdfb01046e0 con 0x7fdfc0102680 2024-08-20T06:36:57.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.388+0000 7fdfbe59c640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 0x7fdfa80b6090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009b40 tx=0x7fdfb4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.390+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdfb00cbcf0 con 0x7fdfc0102680 2024-08-20T06:36:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:57 smithi193 ceph-mon[21944]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:36:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T06:36:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:57 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:36:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:57 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1460653028' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:57.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.566+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdfc0109a40 con 0x7fdfc0102680 2024-08-20T06:36:57.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.566+0000 7fdfa7fff640 1 -- 172.21.15.107:0/2373356933 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) ==== 104+0+11 (secure 0 0 0) 0x7fdfb00cb5f0 con 0x7fdfc0102680 2024-08-20T06:36:57.567 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-08-20T06:36:57.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 msgr2=0x7fdfa80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 0x7fdfa80b6090 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009b40 tx=0x7fdfb4005e50 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 msgr2=0x7fdfc0198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0005ab0 tx=0x7fdfb00042f0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 shutdown_connections 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdfa80b3bd0 0x7fdfa80b6090 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdfc0104190 0x7fdfc0199c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdfc01037d0 0x7fdfc0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 --2- 172.21.15.107:0/2373356933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdfc0102680 0x7fdfc0198980 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.568+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 >> 172.21.15.107:0/2373356933 conn(0x7fdfc00fdf40 msgr2=0x7fdfc0106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.569+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 shutdown_connections 2024-08-20T06:36:57.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.569+0000 7fdfc5085640 1 -- 172.21.15.107:0/2373356933 wait complete. 2024-08-20T06:36:57.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:57 smithi107 ceph-mon[18495]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:36:57.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T06:36:57.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:57 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:36:57.579 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:57 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1460653028' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:57.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-20T06:36:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 -- 172.21.15.107:0/1108961982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c101580 msgr2=0x7f209c101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/1108961982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c101580 0x7f209c101980 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2090009930 tx=0x7f209002a420 comp rx=0 tx=0).stop 2024-08-20T06:36:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 -- 172.21.15.107:0/1108961982 shutdown_connections 2024-08-20T06:36:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/1108961982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c103090 0x7f209c109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/1108961982 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c1026d0 0x7f209c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/1108961982 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c101580 0x7f209c101980 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 -- 172.21.15.107:0/1108961982 >> 172.21.15.107:0/1108961982 conn(0x7f209c0fce40 msgr2=0x7f209c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 -- 172.21.15.107:0/1108961982 shutdown_connections 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.680+0000 7f20a2fd9640 1 -- 172.21.15.107:0/1108961982 wait complete. 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 Processor -- start 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 -- start start 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c101580 0x7f209c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 0x7f209c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10d080 con 0x7f209c1026d0 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10cf00 con 0x7f209c103090 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a2fd9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10d200 con 0x7f209c101580 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60188/0 (socket says 172.21.15.107:60188) 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 -- 172.21.15.107:0/2480557117 learned_addr learned my addr 172.21.15.107:0/2480557117 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a0d75640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c101580 0x7f209c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 -- 172.21.15.107:0/2480557117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c101580 msgr2=0x7f209c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c101580 0x7f209c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 -- 172.21.15.107:0/2480557117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 msgr2=0x7f209c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a1576640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 0x7f209c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 0x7f209c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 -- 172.21.15.107:0/2480557117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c1993e0 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f20a1576640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 0x7f209c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.681+0000 7f209bfff640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f208800c890 tx=0x7f208800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.682+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088004220 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.682+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2088007e40 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.682+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2088004a00 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.682+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c1996d0 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.682+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c19fda0 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.683+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c108970 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.684+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 18) ==== 151060+0+0 (secure 0 0 0) 0x7f20880079b0 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.685+0000 7f2099ffb640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 0x7f208c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.685+0000 7f20a0d75640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 0x7f208c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.685+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f20880e3980 con 0x7f209c1026d0 2024-08-20T06:36:57.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.685+0000 7f20a0d75640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 0x7f208c0b6090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f20900057c0 tx=0x7f2090031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:57.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.689+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2088014030 con 0x7f209c1026d0 2024-08-20T06:36:57.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:57 smithi183 ceph-mon[22204]: pgmap v330: 97 pgs: 97 active+clean; 641 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:36:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3715736242' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-08-20T06:36:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:57 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:36:57.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:57 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1460653028' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:36:57.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.863+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f209c063a80 con 0x7f209c1026d0 2024-08-20T06:36:57.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.864+0000 7f2099ffb640 1 -- 172.21.15.107:0/2480557117 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) ==== 100+0+11 (secure 0 0 0) 0x7f20880aaf90 con 0x7f209c1026d0 2024-08-20T06:36:57.865 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 msgr2=0x7f208c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 0x7f208c0b6090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f20900057c0 tx=0x7f2090031400 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 msgr2=0x7f209c197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f208800c890 tx=0x7f208800cd50 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 shutdown_connections 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f209c103090 0x7f209c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f209c1026d0 0x7f209c197ef0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f208c0b3bd0 0x7f208c0b6090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 --2- 172.21.15.107:0/2480557117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f209c101580 0x7f209c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:57.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 >> 172.21.15.107:0/2480557117 conn(0x7f209c0fce40 msgr2=0x7f209c105b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:57.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 shutdown_connections 2024-08-20T06:36:57.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:57.866+0000 7f20a2fd9640 1 -- 172.21.15.107:0/2480557117 wait complete. 2024-08-20T06:36:57.877 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-08-20T06:36:57.877 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:36:58.125 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:36:58.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:58 smithi193 ceph-mon[21944]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/605173705' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2373356933' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:58 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2480557117' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.562 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:58 smithi107 ceph-mon[18495]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:58.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/605173705' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2373356933' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.563 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:58 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2480557117' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.642+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2147068907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d3010ce40 msgr2=0x7f4d3010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.642+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2147068907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d3010ce40 0x7f4d3010f2a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4d28009ca0 tx=0x7f4d2802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.642+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2147068907 shutdown_connections 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.642+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2147068907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d3010ce40 0x7f4d3010f2a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2147068907 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d30104300 0x7f4d3010c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.663 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2147068907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d301039e0 0x7f4d30103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2147068907 >> 172.21.15.107:0/2147068907 conn(0x7f4d300fd440 msgr2=0x7f4d300ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2147068907 shutdown_connections 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2147068907 wait complete. 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 Processor -- start 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.643+0000 7f4d34a4e640 1 -- start start 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 0x7f4d30101bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 0x7f4d30102110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30112a90 con 0x7f4d301039e0 2024-08-20T06:36:58.664 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30112910 con 0x7f4d3010ce40 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30112c10 con 0x7f4d30104300 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60200/0 (socket says 172.21.15.107:60200) 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 -- 172.21.15.107:0/2012222866 learned_addr learned my addr 172.21.15.107:0/2012222866 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2ed9d640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 0x7f4d30102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d27fff640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 0x7f4d30101bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 -- 172.21.15.107:0/2012222866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 msgr2=0x7f4d30101bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 0x7f4d30101bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.665 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 -- 172.21.15.107:0/2012222866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 msgr2=0x7f4d30102110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 0x7f4d30102110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 -- 172.21.15.107:0/2012222866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d301a7380 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2ed9d640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 0x7f4d30102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d27fff640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 0x7f4d30101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d2e59c640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000b4a0 tx=0x7f4d2000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20014070 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4d20010aa0 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d301a7670 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d20017b70 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.644+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d301a7be0 con 0x7f4d301039e0 2024-08-20T06:36:58.666 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.646+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d30103dc0 con 0x7f4d301039e0 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.647+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f4d20017070 con 0x7f4d301039e0 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.647+0000 7f4d277fe640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 0x7f4d1c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.648+0000 7f4d27fff640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 0x7f4d1c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.648+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4d200e34c0 con 0x7f4d301039e0 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.648+0000 7f4d27fff640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 0x7f4d1c0b6120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006fd0 tx=0x7f4d18008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:58.667 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.651+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4d200aaac0 con 0x7f4d301039e0 2024-08-20T06:36:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:58 smithi183 ceph-mon[22204]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T06:36:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/605173705' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2373356933' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:58 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2480557117' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:36:58.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.862+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d301a8000 con 0x7f4d301039e0 2024-08-20T06:36:58.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.863+0000 7f4d277fe640 1 -- 172.21.15.107:0/2012222866 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7f4d200aa3c0 con 0x7f4d301039e0 2024-08-20T06:36:58.864 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:58.864 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"btime":"2024-08-20T06:36:56:241712+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:36:56.009554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:36:58.864 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 msgr2=0x7f4d1c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 0x7f4d1c0b6120 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d18006fd0 tx=0x7f4d18008040 comp rx=0 tx=0).stop 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 msgr2=0x7f4d30101690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000b4a0 tx=0x7f4d2000b960 comp rx=0 tx=0).stop 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 shutdown_connections 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4d3010ce40 0x7f4d30102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4d1c0b3c60 0x7f4d1c0b6120 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d30104300 0x7f4d30101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 --2- 172.21.15.107:0/2012222866 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4d301039e0 0x7f4d30101690 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:58.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 >> 172.21.15.107:0/2012222866 conn(0x7f4d300fd440 msgr2=0x7f4d300fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:58.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 shutdown_connections 2024-08-20T06:36:58.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:58.865+0000 7f4d34a4e640 1 -- 172.21.15.107:0/2012222866 wait complete. 2024-08-20T06:36:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:59 smithi193 ceph-mon[21944]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:36:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:59 smithi193 ceph-mon[21944]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-20T06:36:59.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:36:59 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2012222866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:59.592 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:59 smithi107 ceph-mon[18495]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:36:59.592 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:59 smithi107 ceph-mon[18495]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-20T06:36:59.592 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:36:59 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2012222866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:59.620 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-08-20T06:36:59.620 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-08-20T06:36:59.620 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T06:36:59.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 -- 172.21.15.107:0/2476584355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 msgr2=0x7fbe6c102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:59.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 --2- 172.21.15.107:0/2476584355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c102aa0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009930 tx=0x7fbe5c02a420 comp rx=0 tx=0).stop 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 -- 172.21.15.107:0/2476584355 shutdown_connections 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 --2- 172.21.15.107:0/2476584355 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6c1041b0 0x7fbe6c10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 --2- 172.21.15.107:0/2476584355 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 0x7fbe6c103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 --2- 172.21.15.107:0/2476584355 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c102aa0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 -- 172.21.15.107:0/2476584355 >> 172.21.15.107:0/2476584355 conn(0x7fbe6c0fdf80 msgr2=0x7fbe6c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 -- 172.21.15.107:0/2476584355 shutdown_connections 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.722+0000 7fbe726dc640 1 -- 172.21.15.107:0/2476584355 wait complete. 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 Processor -- start 2024-08-20T06:36:59.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 -- start start 2024-08-20T06:36:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 0x7fbe6c198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6c1041b0 0x7fbe6c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:59.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10e1f0 con 0x7fbe6c1026a0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10e070 con 0x7fbe6c1041b0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe726dc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6c10e370 con 0x7fbe6c1037f0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60206/0 (socket says 172.21.15.107:60206) 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 -- 172.21.15.107:0/3179418832 learned_addr learned my addr 172.21.15.107:0/3179418832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 -- 172.21.15.107:0/3179418832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 msgr2=0x7fbe6c198f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6b7fe640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 0x7fbe6c198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 0x7fbe6c198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 -- 172.21.15.107:0/3179418832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6c1041b0 msgr2=0x7fbe6c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6c1041b0 0x7fbe6c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 -- 172.21.15.107:0/3179418832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6c19a430 con 0x7fbe6c1026a0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.723+0000 7fbe6bfff640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c031460 tx=0x7fbe5c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:59.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.724+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c03f070 con 0x7fbe6c1026a0 2024-08-20T06:36:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.724+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fbe5c031da0 con 0x7fbe6c1026a0 2024-08-20T06:36:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.724+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6c19a6c0 con 0x7fbe6c1026a0 2024-08-20T06:36:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.724+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c035030 con 0x7fbe6c1026a0 2024-08-20T06:36:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.724+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe6c1a0df0 con 0x7fbe6c1026a0 2024-08-20T06:36:59.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.725+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6c1099b0 con 0x7fbe6c1026a0 2024-08-20T06:36:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.726+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7fbe5c042070 con 0x7fbe6c1026a0 2024-08-20T06:36:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.727+0000 7fbe697fa640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 0x7fbe540b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:36:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.727+0000 7fbe6b7fe640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 0x7fbe540b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:36:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.727+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fbe5c104d50 con 0x7fbe6c1026a0 2024-08-20T06:36:59.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.727+0000 7fbe6b7fe640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 0x7fbe540b61a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ba40 tx=0x7fbe60005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:36:59.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.730+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbe5c0cc200 con 0x7fbe6c1026a0 2024-08-20T06:36:59.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:59 smithi183 ceph-mon[22204]: pgmap v331: 97 pgs: 97 active+clean; 646 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:36:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:59 smithi183 ceph-mon[22204]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-08-20T06:36:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:36:59 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2012222866' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:36:59.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.903+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe6c0632c0 con 0x7fbe540b3ce0 2024-08-20T06:36:59.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.907+0000 7fbe697fa640 1 -- 172.21.15.107:0/3179418832 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258845 (secure 0 0 0) 0x7fbe6c0632c0 con 0x7fbe540b3ce0 2024-08-20T06:36:59.911 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:36:59.913 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:36:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.910+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 msgr2=0x7fbe540b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.910+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 0x7fbe540b61a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ba40 tx=0x7fbe60005cc0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 msgr2=0x7fbe6c1988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:36:59.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c031460 tx=0x7fbe5c003c80 comp rx=0 tx=0).stop 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 shutdown_connections 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fbe6c1041b0 0x7fbe6c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fbe540b3ce0 0x7fbe540b61a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe6c1037f0 0x7fbe6c198f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 --2- 172.21.15.107:0/3179418832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbe6c1026a0 0x7fbe6c1988f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 >> 172.21.15.107:0/3179418832 conn(0x7fbe6c0fdf80 msgr2=0x7fbe6c106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 shutdown_connections 2024-08-20T06:36:59.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:36:59.911+0000 7fbe726dc640 1 -- 172.21.15.107:0/3179418832 wait complete. 2024-08-20T06:36:59.922 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-08-20T06:36:59.334749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662786,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":472,"num_write":299,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369072,"kb_used_data":11504,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572752,"statfs":{"total":1151940427776,"available":1151562498048,"internally_reserved":0,"allocated":11780096,"data_stored":6603933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":39,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"3.3f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.524915+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:31.524915+0000","last_peered":"2024-08-20T06:36:31.524915+0000","last_clean":"2024-08-20T06:36:31.524915+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:31.524915+0000","last_undegraded":"2024-08-20T06:36:31.524915+0000","last_fullsized":"2024-08-20T06:36:31.524915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534240+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:31.534240+0000","last_peered":"2024-08-20T06:36:31.534240+0000","last_clean":"2024-08-20T06:36:31.534240+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:31.534240+0000","last_undegraded":"2024-08-20T06:36:31.534240+0000","last_fullsized":"2024-08-20T06:36:31.534240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540252+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:31.540252+0000","last_peered":"2024-08-20T06:36:31.540252+0000","last_clean":"2024-08-20T06:36:31.540252+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:31.540252+0000","last_undegraded":"2024-08-20T06:36:31.540252+0000","last_fullsized":"2024-08-20T06:36:31.540252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.515382+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:31.515382+0000","last_peered":"2024-08-20T06:36:31.515382+0000","last_clean":"2024-08-20T06:36:31.515382+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:31.515382+0000","last_undegraded":"2024-08-20T06:36:31.515382+0000","last_fullsized":"2024-08-20T06:36:31.515382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.514978+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:31.514978+0000","last_peered":"2024-08-20T06:36:31.514978+0000","last_clean":"2024-08-20T06:36:31.514978+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:31.514978+0000","last_undegraded":"2024-08-20T06:36:31.514978+0000","last_fullsized":"2024-08-20T06:36:31.514978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537525+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:31.537525+0000","last_peered":"2024-08-20T06:36:31.537525+0000","last_clean":"2024-08-20T06:36:31.537525+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:31.537525+0000","last_undegraded":"2024-08-20T06:36:31.537525+0000","last_fullsized":"2024-08-20T06:36:31.537525+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"73'5","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528038+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:31.528038+0000","last_peered":"2024-08-20T06:36:31.528038+0000","last_clean":"2024-08-20T06:36:31.528038+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:31.528038+0000","last_undegraded":"2024-08-20T06:36:31.528038+0000","last_fullsized":"2024-08-20T06:36:31.528038+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535771+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:31.535771+0000","last_peered":"2024-08-20T06:36:31.535771+0000","last_clean":"2024-08-20T06:36:31.535771+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:31.535771+0000","last_undegraded":"2024-08-20T06:36:31.535771+0000","last_fullsized":"2024-08-20T06:36:31.535771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.543127+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:31.543127+0000","last_peered":"2024-08-20T06:36:31.543127+0000","last_clean":"2024-08-20T06:36:31.543127+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:31.543127+0000","last_undegraded":"2024-08-20T06:36:31.543127+0000","last_fullsized":"2024-08-20T06:36:31.543127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"73'16","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.536670+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:31.536670+0000","last_peered":"2024-08-20T06:36:31.536670+0000","last_clean":"2024-08-20T06:36:31.536670+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:31.536670+0000","last_undegraded":"2024-08-20T06:36:31.536670+0000","last_fullsized":"2024-08-20T06:36:31.536670+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538446+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:31.538446+0000","last_peered":"2024-08-20T06:36:31.538446+0000","last_clean":"2024-08-20T06:36:31.538446+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:31.538446+0000","last_undegraded":"2024-08-20T06:36:31.538446+0000","last_fullsized":"2024-08-20T06:36:31.538446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541569+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:31.541569+0000","last_peered":"2024-08-20T06:36:31.541569+0000","last_clean":"2024-08-20T06:36:31.541569+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:31.541569+0000","last_undegraded":"2024-08-20T06:36:31.541569+0000","last_fullsized":"2024-08-20T06:36:31.541569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530179+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:31.530179+0000","last_peered":"2024-08-20T06:36:31.530179+0000","last_clean":"2024-08-20T06:36:31.530179+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:31.530179+0000","last_undegraded":"2024-08-20T06:36:31.530179+0000","last_fullsized":"2024-08-20T06:36:31.530179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534684+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:31.534684+0000","last_peered":"2024-08-20T06:36:31.534684+0000","last_clean":"2024-08-20T06:36:31.534684+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:31.534684+0000","last_undegraded":"2024-08-20T06:36:31.534684+0000","last_fullsized":"2024-08-20T06:36:31.534684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540482+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:31.540482+0000","last_peered":"2024-08-20T06:36:31.540482+0000","last_clean":"2024-08-20T06:36:31.540482+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:31.540482+0000","last_undegraded":"2024-08-20T06:36:31.540482+0000","last_fullsized":"2024-08-20T06:36:31.540482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542459+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:31.542459+0000","last_peered":"2024-08-20T06:36:31.542459+0000","last_clean":"2024-08-20T06:36:31.542459+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:31.542459+0000","last_undegraded":"2024-08-20T06:36:31.542459+0000","last_fullsized":"2024-08-20T06:36:31.542459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525439+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:31.525439+0000","last_peered":"2024-08-20T06:36:31.525439+0000","last_clean":"2024-08-20T06:36:31.525439+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:31.525439+0000","last_undegraded":"2024-08-20T06:36:31.525439+0000","last_fullsized":"2024-08-20T06:36:31.525439+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535381+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:31.535381+0000","last_peered":"2024-08-20T06:36:31.535381+0000","last_clean":"2024-08-20T06:36:31.535381+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:31.535381+0000","last_undegraded":"2024-08-20T06:36:31.535381+0000","last_fullsized":"2024-08-20T06:36:31.535381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526562+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:31.526562+0000","last_peered":"2024-08-20T06:36:31.526562+0000","last_clean":"2024-08-20T06:36:31.526562+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:31.526562+0000","last_undegraded":"2024-08-20T06:36:31.526562+0000","last_fullsized":"2024-08-20T06:36:31.526562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527011+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:31.527011+0000","last_peered":"2024-08-20T06:36:31.527011+0000","last_clean":"2024-08-20T06:36:31.527011+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:31.527011+0000","last_undegraded":"2024-08-20T06:36:31.527011+0000","last_fullsized":"2024-08-20T06:36:31.527011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.533876+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:31.533876+0000","last_peered":"2024-08-20T06:36:31.533876+0000","last_clean":"2024-08-20T06:36:31.533876+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:31.533876+0000","last_undegraded":"2024-08-20T06:36:31.533876+0000","last_fullsized":"2024-08-20T06:36:31.533876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542900+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:31.542900+0000","last_peered":"2024-08-20T06:36:31.542900+0000","last_clean":"2024-08-20T06:36:31.542900+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:31.542900+0000","last_undegraded":"2024-08-20T06:36:31.542900+0000","last_fullsized":"2024-08-20T06:36:31.542900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523834+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:31.523834+0000","last_peered":"2024-08-20T06:36:31.523834+0000","last_clean":"2024-08-20T06:36:31.523834+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:31.523834+0000","last_undegraded":"2024-08-20T06:36:31.523834+0000","last_fullsized":"2024-08-20T06:36:31.523834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529739+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:31.529739+0000","last_peered":"2024-08-20T06:36:31.529739+0000","last_clean":"2024-08-20T06:36:31.529739+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:31.529739+0000","last_undegraded":"2024-08-20T06:36:31.529739+0000","last_fullsized":"2024-08-20T06:36:31.529739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.521121+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:31.521121+0000","last_peered":"2024-08-20T06:36:31.521121+0000","last_clean":"2024-08-20T06:36:31.521121+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:31.521121+0000","last_undegraded":"2024-08-20T06:36:31.521121+0000","last_fullsized":"2024-08-20T06:36:31.521121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529168+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:31.529168+0000","last_peered":"2024-08-20T06:36:31.529168+0000","last_clean":"2024-08-20T06:36:31.529168+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:31.529168+0000","last_undegraded":"2024-08-20T06:36:31.529168+0000","last_fullsized":"2024-08-20T06:36:31.529168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526180+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:31.526180+0000","last_peered":"2024-08-20T06:36:31.526180+0000","last_clean":"2024-08-20T06:36:31.526180+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:31.526180+0000","last_undegraded":"2024-08-20T06:36:31.526180+0000","last_fullsized":"2024-08-20T06:36:31.526180+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.522146+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:31.522146+0000","last_peered":"2024-08-20T06:36:31.522146+0000","last_clean":"2024-08-20T06:36:31.522146+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:31.522146+0000","last_undegraded":"2024-08-20T06:36:31.522146+0000","last_fullsized":"2024-08-20T06:36:31.522146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540463+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:31.540463+0000","last_peered":"2024-08-20T06:36:31.540463+0000","last_clean":"2024-08-20T06:36:31.540463+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:31.540463+0000","last_undegraded":"2024-08-20T06:36:31.540463+0000","last_fullsized":"2024-08-20T06:36:31.540463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.520194+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:31.520194+0000","last_peered":"2024-08-20T06:36:31.520194+0000","last_clean":"2024-08-20T06:36:31.520194+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:31.520194+0000","last_undegraded":"2024-08-20T06:36:31.520194+0000","last_fullsized":"2024-08-20T06:36:31.520194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"73'12","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530539+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:31.530539+0000","last_peered":"2024-08-20T06:36:31.530539+0000","last_clean":"2024-08-20T06:36:31.530539+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:31.530539+0000","last_undegraded":"2024-08-20T06:36:31.530539+0000","last_fullsized":"2024-08-20T06:36:31.530539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528330+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:31.528330+0000","last_peered":"2024-08-20T06:36:31.528330+0000","last_clean":"2024-08-20T06:36:31.528330+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:31.528330+0000","last_undegraded":"2024-08-20T06:36:31.528330+0000","last_fullsized":"2024-08-20T06:36:31.528330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529088+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:31.529088+0000","last_peered":"2024-08-20T06:36:31.529088+0000","last_clean":"2024-08-20T06:36:31.529088+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:31.529088+0000","last_undegraded":"2024-08-20T06:36:31.529088+0000","last_fullsized":"2024-08-20T06:36:31.529088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'6","reported_seq":138,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:54.144297+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:54.144297+0000","last_peered":"2024-08-20T06:36:54.144297+0000","last_clean":"2024-08-20T06:36:54.144297+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:54.144297+0000","last_undegraded":"2024-08-20T06:36:54.144297+0000","last_fullsized":"2024-08-20T06:36:54.144297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541085+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:31.541085+0000","last_peered":"2024-08-20T06:36:31.541085+0000","last_clean":"2024-08-20T06:36:31.541085+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:31.541085+0000","last_undegraded":"2024-08-20T06:36:31.541085+0000","last_fullsized":"2024-08-20T06:36:31.541085+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226778+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:03.226778+0000","last_peered":"2024-08-20T06:36:03.226778+0000","last_clean":"2024-08-20T06:36:03.226778+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:03.226778+0000","last_undegraded":"2024-08-20T06:36:03.226778+0000","last_fullsized":"2024-08-20T06:36:03.226778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523940+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:31.523940+0000","last_peered":"2024-08-20T06:36:31.523940+0000","last_clean":"2024-08-20T06:36:31.523940+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:31.523940+0000","last_undegraded":"2024-08-20T06:36:31.523940+0000","last_fullsized":"2024-08-20T06:36:31.523940+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534641+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:31.534641+0000","last_peered":"2024-08-20T06:36:31.534641+0000","last_clean":"2024-08-20T06:36:31.534641+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:31.534641+0000","last_undegraded":"2024-08-20T06:36:31.534641+0000","last_fullsized":"2024-08-20T06:36:31.534641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'4","reported_seq":135,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.147134+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:56.147134+0000","last_peered":"2024-08-20T06:36:56.147134+0000","last_clean":"2024-08-20T06:36:56.147134+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:56.147134+0000","last_undegraded":"2024-08-20T06:36:56.147134+0000","last_fullsized":"2024-08-20T06:36:56.147134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541403+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:31.541403+0000","last_peered":"2024-08-20T06:36:31.541403+0000","last_clean":"2024-08-20T06:36:31.541403+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:31.541403+0000","last_undegraded":"2024-08-20T06:36:31.541403+0000","last_fullsized":"2024-08-20T06:36:31.541403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"73'9","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531015+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:31.531015+0000","last_peered":"2024-08-20T06:36:31.531015+0000","last_clean":"2024-08-20T06:36:31.531015+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:31.531015+0000","last_undegraded":"2024-08-20T06:36:31.531015+0000","last_fullsized":"2024-08-20T06:36:31.531015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542985+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:31.542985+0000","last_peered":"2024-08-20T06:36:31.542985+0000","last_clean":"2024-08-20T06:36:31.542985+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:31.542985+0000","last_undegraded":"2024-08-20T06:36:31.542985+0000","last_fullsized":"2024-08-20T06:36:31.542985+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"73'35","reported_seq":301,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.428964+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:56.428964+0000","last_peered":"2024-08-20T06:36:56.428964+0000","last_clean":"2024-08-20T06:36:56.428964+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:56.428964+0000","last_undegraded":"2024-08-20T06:36:56.428964+0000","last_fullsized":"2024-08-20T06:36:56.428964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67197,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531845+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:31.531845+0000","last_peered":"2024-08-20T06:36:31.531845+0000","last_clean":"2024-08-20T06:36:31.531845+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:31.531845+0000","last_undegraded":"2024-08-20T06:36:31.531845+0000","last_fullsized":"2024-08-20T06:36:31.531845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532459+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:31.532459+0000","last_peered":"2024-08-20T06:36:31.532459+0000","last_clean":"2024-08-20T06:36:31.532459+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:31.532459+0000","last_undegraded":"2024-08-20T06:36:31.532459+0000","last_fullsized":"2024-08-20T06:36:31.532459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"73'15","reported_seq":40,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534636+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:31.534636+0000","last_peered":"2024-08-20T06:36:31.534636+0000","last_clean":"2024-08-20T06:36:31.534636+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:31.534636+0000","last_undegraded":"2024-08-20T06:36:31.534636+0000","last_fullsized":"2024-08-20T06:36:31.534636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'5","reported_seq":140,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:57.159656+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:57.159656+0000","last_peered":"2024-08-20T06:36:57.159656+0000","last_clean":"2024-08-20T06:36:57.159656+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:57.159656+0000","last_undegraded":"2024-08-20T06:36:57.159656+0000","last_fullsized":"2024-08-20T06:36:57.159656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532941+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:31.532941+0000","last_peered":"2024-08-20T06:36:31.532941+0000","last_clean":"2024-08-20T06:36:31.532941+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:31.532941+0000","last_undegraded":"2024-08-20T06:36:31.532941+0000","last_fullsized":"2024-08-20T06:36:31.532941+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542040+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:31.542040+0000","last_peered":"2024-08-20T06:36:31.542040+0000","last_clean":"2024-08-20T06:36:31.542040+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:31.542040+0000","last_undegraded":"2024-08-20T06:36:31.542040+0000","last_fullsized":"2024-08-20T06:36:31.542040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527503+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:31.527503+0000","last_peered":"2024-08-20T06:36:31.527503+0000","last_clean":"2024-08-20T06:36:31.527503+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:31.527503+0000","last_undegraded":"2024-08-20T06:36:31.527503+0000","last_fullsized":"2024-08-20T06:36:31.527503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227139+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:03.227139+0000","last_peered":"2024-08-20T06:36:03.227139+0000","last_clean":"2024-08-20T06:36:03.227139+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:03.227139+0000","last_undegraded":"2024-08-20T06:36:03.227139+0000","last_fullsized":"2024-08-20T06:36:03.227139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530907+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:31.530907+0000","last_peered":"2024-08-20T06:36:31.530907+0000","last_clean":"2024-08-20T06:36:31.530907+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:31.530907+0000","last_undegraded":"2024-08-20T06:36:31.530907+0000","last_fullsized":"2024-08-20T06:36:31.530907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'7","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531455+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:31.531455+0000","last_peered":"2024-08-20T06:36:31.531455+0000","last_clean":"2024-08-20T06:36:31.531455+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:31.531455+0000","last_undegraded":"2024-08-20T06:36:31.531455+0000","last_fullsized":"2024-08-20T06:36:31.531455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538900+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:31.538900+0000","last_peered":"2024-08-20T06:36:31.538900+0000","last_clean":"2024-08-20T06:36:31.538900+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:31.538900+0000","last_undegraded":"2024-08-20T06:36:31.538900+0000","last_fullsized":"2024-08-20T06:36:31.538900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":92,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122684+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122684+0000","last_peered":"2024-08-20T06:36:03.122684+0000","last_clean":"2024-08-20T06:36:03.122684+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122684+0000","last_undegraded":"2024-08-20T06:36:03.122684+0000","last_fullsized":"2024-08-20T06:36:03.122684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339216+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:03.339216+0000","last_peered":"2024-08-20T06:36:03.339216+0000","last_clean":"2024-08-20T06:36:03.339216+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:03.339216+0000","last_undegraded":"2024-08-20T06:36:03.339216+0000","last_fullsized":"2024-08-20T06:36:03.339216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529343+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:31.529343+0000","last_peered":"2024-08-20T06:36:31.529343+0000","last_clean":"2024-08-20T06:36:31.529343+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:31.529343+0000","last_undegraded":"2024-08-20T06:36:31.529343+0000","last_fullsized":"2024-08-20T06:36:31.529343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.135010+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.135010+0000","last_peered":"2024-08-20T06:36:03.135010+0000","last_clean":"2024-08-20T06:36:03.135010+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.135010+0000","last_undegraded":"2024-08-20T06:36:03.135010+0000","last_fullsized":"2024-08-20T06:36:03.135010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525687+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:31.525687+0000","last_peered":"2024-08-20T06:36:31.525687+0000","last_clean":"2024-08-20T06:36:31.525687+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:31.525687+0000","last_undegraded":"2024-08-20T06:36:31.525687+0000","last_fullsized":"2024-08-20T06:36:31.525687+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'5","reported_seq":136,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.157371+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:56.157371+0000","last_peered":"2024-08-20T06:36:56.157371+0000","last_clean":"2024-08-20T06:36:56.157371+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:56.157371+0000","last_undegraded":"2024-08-20T06:36:56.157371+0000","last_fullsized":"2024-08-20T06:36:56.157371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541026+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:31.541026+0000","last_peered":"2024-08-20T06:36:31.541026+0000","last_clean":"2024-08-20T06:36:31.541026+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:31.541026+0000","last_undegraded":"2024-08-20T06:36:31.541026+0000","last_fullsized":"2024-08-20T06:36:31.541026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530300+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:31.530300+0000","last_peered":"2024-08-20T06:36:31.530300+0000","last_clean":"2024-08-20T06:36:31.530300+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:31.530300+0000","last_undegraded":"2024-08-20T06:36:31.530300+0000","last_fullsized":"2024-08-20T06:36:31.530300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537954+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:31.537954+0000","last_peered":"2024-08-20T06:36:31.537954+0000","last_clean":"2024-08-20T06:36:31.537954+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:31.537954+0000","last_undegraded":"2024-08-20T06:36:31.537954+0000","last_fullsized":"2024-08-20T06:36:31.537954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526170+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:31.526170+0000","last_peered":"2024-08-20T06:36:31.526170+0000","last_clean":"2024-08-20T06:36:31.526170+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:31.526170+0000","last_undegraded":"2024-08-20T06:36:31.526170+0000","last_fullsized":"2024-08-20T06:36:31.526170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'11","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123684+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123684+0000","last_peered":"2024-08-20T06:36:03.123684+0000","last_clean":"2024-08-20T06:36:03.123684+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123684+0000","last_undegraded":"2024-08-20T06:36:03.123684+0000","last_fullsized":"2024-08-20T06:36:03.123684+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525153+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:31.525153+0000","last_peered":"2024-08-20T06:36:31.525153+0000","last_clean":"2024-08-20T06:36:31.525153+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:31.525153+0000","last_undegraded":"2024-08-20T06:36:31.525153+0000","last_fullsized":"2024-08-20T06:36:31.525153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339652+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:03.339652+0000","last_peered":"2024-08-20T06:36:03.339652+0000","last_clean":"2024-08-20T06:36:03.339652+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:03.339652+0000","last_undegraded":"2024-08-20T06:36:03.339652+0000","last_fullsized":"2024-08-20T06:36:03.339652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540510+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:31.540510+0000","last_peered":"2024-08-20T06:36:31.540510+0000","last_clean":"2024-08-20T06:36:31.540510+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:31.540510+0000","last_undegraded":"2024-08-20T06:36:31.540510+0000","last_fullsized":"2024-08-20T06:36:31.540510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.493499+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:31.493499+0000","last_peered":"2024-08-20T06:36:31.493499+0000","last_clean":"2024-08-20T06:36:31.493499+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:31.493499+0000","last_undegraded":"2024-08-20T06:36:31.493499+0000","last_fullsized":"2024-08-20T06:36:31.493499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124496+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:03.124496+0000","last_peered":"2024-08-20T06:36:03.124496+0000","last_clean":"2024-08-20T06:36:03.124496+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:03.124496+0000","last_undegraded":"2024-08-20T06:36:03.124496+0000","last_fullsized":"2024-08-20T06:36:03.124496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527640+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:31.527640+0000","last_peered":"2024-08-20T06:36:31.527640+0000","last_clean":"2024-08-20T06:36:31.527640+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:31.527640+0000","last_undegraded":"2024-08-20T06:36:31.527640+0000","last_fullsized":"2024-08-20T06:36:31.527640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535568+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:31.535568+0000","last_peered":"2024-08-20T06:36:31.535568+0000","last_clean":"2024-08-20T06:36:31.535568+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:31.535568+0000","last_undegraded":"2024-08-20T06:36:31.535568+0000","last_fullsized":"2024-08-20T06:36:31.535568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.539026+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:31.539026+0000","last_peered":"2024-08-20T06:36:31.539026+0000","last_clean":"2024-08-20T06:36:31.539026+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:31.539026+0000","last_undegraded":"2024-08-20T06:36:31.539026+0000","last_fullsized":"2024-08-20T06:36:31.539026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542524+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:31.542524+0000","last_peered":"2024-08-20T06:36:31.542524+0000","last_clean":"2024-08-20T06:36:31.542524+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:31.542524+0000","last_undegraded":"2024-08-20T06:36:31.542524+0000","last_fullsized":"2024-08-20T06:36:31.542524+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541846+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:31.541846+0000","last_peered":"2024-08-20T06:36:31.541846+0000","last_clean":"2024-08-20T06:36:31.541846+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:31.541846+0000","last_undegraded":"2024-08-20T06:36:31.541846+0000","last_fullsized":"2024-08-20T06:36:31.541846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541363+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:31.541363+0000","last_peered":"2024-08-20T06:36:31.541363+0000","last_clean":"2024-08-20T06:36:31.541363+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:31.541363+0000","last_undegraded":"2024-08-20T06:36:31.541363+0000","last_fullsized":"2024-08-20T06:36:31.541363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125324+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:03.125324+0000","last_peered":"2024-08-20T06:36:03.125324+0000","last_clean":"2024-08-20T06:36:03.125324+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:03.125324+0000","last_undegraded":"2024-08-20T06:36:03.125324+0000","last_fullsized":"2024-08-20T06:36:03.125324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528633+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:31.528633+0000","last_peered":"2024-08-20T06:36:31.528633+0000","last_clean":"2024-08-20T06:36:31.528633+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:31.528633+0000","last_undegraded":"2024-08-20T06:36:31.528633+0000","last_fullsized":"2024-08-20T06:36:31.528633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72184,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":672,"num_read_kb":408,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037801,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1045241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168622,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332148,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528376,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1409024,"data_stored":978360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":802816,"data_stored":385304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1421312,"data_stored":977551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051655,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":453822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607634,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":447589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":386575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:36:59.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T06:37:00.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2951852402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341030a0 msgr2=0x7f1d34109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2951852402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341030a0 0x7f1d34109a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009cd0 tx=0x7f1d2c02a8f0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2951852402 shutdown_connections 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2951852402 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341030a0 0x7f1d34109a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2951852402 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d341026e0 0x7f1d34102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2951852402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d34101590 0x7f1d34101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2951852402 >> 172.21.15.107:0/2951852402 conn(0x7f1d340fce50 msgr2=0x7f1d340ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2951852402 shutdown_connections 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.061+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2951852402 wait complete. 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 Processor -- start 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 -- start start 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d34101590 0x7f1d34198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 0x7f1d34199b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3410d0e0 con 0x7f1d341026e0 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3410cf60 con 0x7f1d34101590 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3c49a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3410d260 con 0x7f1d341030a0 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60210/0 (socket says 172.21.15.107:60210) 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 -- 172.21.15.107:0/2664061203 learned_addr learned my addr 172.21.15.107:0/2664061203 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3aa37640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 0x7f1d34199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3a236640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d34101590 0x7f1d34198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 -- 172.21.15.107:0/2664061203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 msgr2=0x7f1d34199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 0x7f1d34199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 -- 172.21.15.107:0/2664061203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d34101590 msgr2=0x7f1d34198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d34101590 0x7f1d34198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d39a35640 1 -- 172.21.15.107:0/2664061203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3419a440 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.062+0000 7f1d3aa37640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 0x7f1d34199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d39a35640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800d950 tx=0x7f1d2800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28017070 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1d28009c30 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3419a730 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2800f620 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.063+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d341a0e00 con 0x7f1d341026e0 2024-08-20T06:37:00.067 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.064+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d341a0fd0 con 0x7f1d341026e0 2024-08-20T06:37:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.065+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f1d28010460 con 0x7f1d341026e0 2024-08-20T06:37:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.066+0000 7f1d277fe640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 0x7f1d200b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.066+0000 7f1d3a236640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 0x7f1d200b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.066+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f1d280e3ab0 con 0x7f1d341026e0 2024-08-20T06:37:00.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.066+0000 7f1d3a236640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 0x7f1d200b6120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d30005e10 tx=0x7f1d30005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:00.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.069+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1d280aafa0 con 0x7f1d341026e0 2024-08-20T06:37:00.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.239+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d340feec0 con 0x7f1d200b3c60 2024-08-20T06:37:00.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.243+0000 7f1d277fe640 1 -- 172.21.15.107:0/2664061203 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258845 (secure 0 0 0) 0x7f1d340feec0 con 0x7f1d200b3c60 2024-08-20T06:37:00.247 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:37:00.248 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:37:00.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 msgr2=0x7f1d200b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 0x7f1d200b6120 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1d30005e10 tx=0x7f1d30005da0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 msgr2=0x7f1d34198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800d950 tx=0x7f1d2800de10 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 shutdown_connections 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1d200b3c60 0x7f1d200b6120 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d341030a0 0x7f1d34199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1d341026e0 0x7f1d34198f50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 --2- 172.21.15.107:0/2664061203 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1d34101590 0x7f1d34198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.246+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 >> 172.21.15.107:0/2664061203 conn(0x7f1d340fce50 msgr2=0x7f1d340fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.247+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 shutdown_connections 2024-08-20T06:37:00.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.247+0000 7f1d3c49a640 1 -- 172.21.15.107:0/2664061203 wait complete. 2024-08-20T06:37:00.259 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-08-20T06:36:59.334749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662786,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":472,"num_write":299,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369072,"kb_used_data":11504,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572752,"statfs":{"total":1151940427776,"available":1151562498048,"internally_reserved":0,"allocated":11780096,"data_stored":6603933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":39,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"3.3f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.524915+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:31.524915+0000","last_peered":"2024-08-20T06:36:31.524915+0000","last_clean":"2024-08-20T06:36:31.524915+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:31.524915+0000","last_undegraded":"2024-08-20T06:36:31.524915+0000","last_fullsized":"2024-08-20T06:36:31.524915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534240+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:31.534240+0000","last_peered":"2024-08-20T06:36:31.534240+0000","last_clean":"2024-08-20T06:36:31.534240+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:31.534240+0000","last_undegraded":"2024-08-20T06:36:31.534240+0000","last_fullsized":"2024-08-20T06:36:31.534240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540252+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:31.540252+0000","last_peered":"2024-08-20T06:36:31.540252+0000","last_clean":"2024-08-20T06:36:31.540252+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:31.540252+0000","last_undegraded":"2024-08-20T06:36:31.540252+0000","last_fullsized":"2024-08-20T06:36:31.540252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.515382+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:31.515382+0000","last_peered":"2024-08-20T06:36:31.515382+0000","last_clean":"2024-08-20T06:36:31.515382+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:31.515382+0000","last_undegraded":"2024-08-20T06:36:31.515382+0000","last_fullsized":"2024-08-20T06:36:31.515382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.514978+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:31.514978+0000","last_peered":"2024-08-20T06:36:31.514978+0000","last_clean":"2024-08-20T06:36:31.514978+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:31.514978+0000","last_undegraded":"2024-08-20T06:36:31.514978+0000","last_fullsized":"2024-08-20T06:36:31.514978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537525+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:31.537525+0000","last_peered":"2024-08-20T06:36:31.537525+0000","last_clean":"2024-08-20T06:36:31.537525+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:31.537525+0000","last_undegraded":"2024-08-20T06:36:31.537525+0000","last_fullsized":"2024-08-20T06:36:31.537525+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"73'5","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528038+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:31.528038+0000","last_peered":"2024-08-20T06:36:31.528038+0000","last_clean":"2024-08-20T06:36:31.528038+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:31.528038+0000","last_undegraded":"2024-08-20T06:36:31.528038+0000","last_fullsized":"2024-08-20T06:36:31.528038+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535771+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:31.535771+0000","last_peered":"2024-08-20T06:36:31.535771+0000","last_clean":"2024-08-20T06:36:31.535771+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:31.535771+0000","last_undegraded":"2024-08-20T06:36:31.535771+0000","last_fullsized":"2024-08-20T06:36:31.535771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.543127+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:31.543127+0000","last_peered":"2024-08-20T06:36:31.543127+0000","last_clean":"2024-08-20T06:36:31.543127+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:31.543127+0000","last_undegraded":"2024-08-20T06:36:31.543127+0000","last_fullsized":"2024-08-20T06:36:31.543127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"73'16","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.536670+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:31.536670+0000","last_peered":"2024-08-20T06:36:31.536670+0000","last_clean":"2024-08-20T06:36:31.536670+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:31.536670+0000","last_undegraded":"2024-08-20T06:36:31.536670+0000","last_fullsized":"2024-08-20T06:36:31.536670+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538446+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:31.538446+0000","last_peered":"2024-08-20T06:36:31.538446+0000","last_clean":"2024-08-20T06:36:31.538446+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:31.538446+0000","last_undegraded":"2024-08-20T06:36:31.538446+0000","last_fullsized":"2024-08-20T06:36:31.538446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541569+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:31.541569+0000","last_peered":"2024-08-20T06:36:31.541569+0000","last_clean":"2024-08-20T06:36:31.541569+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:31.541569+0000","last_undegraded":"2024-08-20T06:36:31.541569+0000","last_fullsized":"2024-08-20T06:36:31.541569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530179+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:31.530179+0000","last_peered":"2024-08-20T06:36:31.530179+0000","last_clean":"2024-08-20T06:36:31.530179+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:31.530179+0000","last_undegraded":"2024-08-20T06:36:31.530179+0000","last_fullsized":"2024-08-20T06:36:31.530179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534684+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:31.534684+0000","last_peered":"2024-08-20T06:36:31.534684+0000","last_clean":"2024-08-20T06:36:31.534684+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:31.534684+0000","last_undegraded":"2024-08-20T06:36:31.534684+0000","last_fullsized":"2024-08-20T06:36:31.534684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540482+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:31.540482+0000","last_peered":"2024-08-20T06:36:31.540482+0000","last_clean":"2024-08-20T06:36:31.540482+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:31.540482+0000","last_undegraded":"2024-08-20T06:36:31.540482+0000","last_fullsized":"2024-08-20T06:36:31.540482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542459+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:31.542459+0000","last_peered":"2024-08-20T06:36:31.542459+0000","last_clean":"2024-08-20T06:36:31.542459+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:31.542459+0000","last_undegraded":"2024-08-20T06:36:31.542459+0000","last_fullsized":"2024-08-20T06:36:31.542459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525439+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:31.525439+0000","last_peered":"2024-08-20T06:36:31.525439+0000","last_clean":"2024-08-20T06:36:31.525439+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:31.525439+0000","last_undegraded":"2024-08-20T06:36:31.525439+0000","last_fullsized":"2024-08-20T06:36:31.525439+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535381+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:31.535381+0000","last_peered":"2024-08-20T06:36:31.535381+0000","last_clean":"2024-08-20T06:36:31.535381+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:31.535381+0000","last_undegraded":"2024-08-20T06:36:31.535381+0000","last_fullsized":"2024-08-20T06:36:31.535381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526562+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:31.526562+0000","last_peered":"2024-08-20T06:36:31.526562+0000","last_clean":"2024-08-20T06:36:31.526562+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:31.526562+0000","last_undegraded":"2024-08-20T06:36:31.526562+0000","last_fullsized":"2024-08-20T06:36:31.526562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527011+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:31.527011+0000","last_peered":"2024-08-20T06:36:31.527011+0000","last_clean":"2024-08-20T06:36:31.527011+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:31.527011+0000","last_undegraded":"2024-08-20T06:36:31.527011+0000","last_fullsized":"2024-08-20T06:36:31.527011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.533876+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:31.533876+0000","last_peered":"2024-08-20T06:36:31.533876+0000","last_clean":"2024-08-20T06:36:31.533876+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:31.533876+0000","last_undegraded":"2024-08-20T06:36:31.533876+0000","last_fullsized":"2024-08-20T06:36:31.533876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542900+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:31.542900+0000","last_peered":"2024-08-20T06:36:31.542900+0000","last_clean":"2024-08-20T06:36:31.542900+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:31.542900+0000","last_undegraded":"2024-08-20T06:36:31.542900+0000","last_fullsized":"2024-08-20T06:36:31.542900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523834+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:31.523834+0000","last_peered":"2024-08-20T06:36:31.523834+0000","last_clean":"2024-08-20T06:36:31.523834+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:31.523834+0000","last_undegraded":"2024-08-20T06:36:31.523834+0000","last_fullsized":"2024-08-20T06:36:31.523834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529739+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:31.529739+0000","last_peered":"2024-08-20T06:36:31.529739+0000","last_clean":"2024-08-20T06:36:31.529739+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:31.529739+0000","last_undegraded":"2024-08-20T06:36:31.529739+0000","last_fullsized":"2024-08-20T06:36:31.529739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.521121+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:31.521121+0000","last_peered":"2024-08-20T06:36:31.521121+0000","last_clean":"2024-08-20T06:36:31.521121+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:31.521121+0000","last_undegraded":"2024-08-20T06:36:31.521121+0000","last_fullsized":"2024-08-20T06:36:31.521121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529168+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:31.529168+0000","last_peered":"2024-08-20T06:36:31.529168+0000","last_clean":"2024-08-20T06:36:31.529168+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:31.529168+0000","last_undegraded":"2024-08-20T06:36:31.529168+0000","last_fullsized":"2024-08-20T06:36:31.529168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526180+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:31.526180+0000","last_peered":"2024-08-20T06:36:31.526180+0000","last_clean":"2024-08-20T06:36:31.526180+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:31.526180+0000","last_undegraded":"2024-08-20T06:36:31.526180+0000","last_fullsized":"2024-08-20T06:36:31.526180+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.522146+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:31.522146+0000","last_peered":"2024-08-20T06:36:31.522146+0000","last_clean":"2024-08-20T06:36:31.522146+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:31.522146+0000","last_undegraded":"2024-08-20T06:36:31.522146+0000","last_fullsized":"2024-08-20T06:36:31.522146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540463+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:31.540463+0000","last_peered":"2024-08-20T06:36:31.540463+0000","last_clean":"2024-08-20T06:36:31.540463+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:31.540463+0000","last_undegraded":"2024-08-20T06:36:31.540463+0000","last_fullsized":"2024-08-20T06:36:31.540463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.520194+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:31.520194+0000","last_peered":"2024-08-20T06:36:31.520194+0000","last_clean":"2024-08-20T06:36:31.520194+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:31.520194+0000","last_undegraded":"2024-08-20T06:36:31.520194+0000","last_fullsized":"2024-08-20T06:36:31.520194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"73'12","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530539+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:31.530539+0000","last_peered":"2024-08-20T06:36:31.530539+0000","last_clean":"2024-08-20T06:36:31.530539+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:31.530539+0000","last_undegraded":"2024-08-20T06:36:31.530539+0000","last_fullsized":"2024-08-20T06:36:31.530539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528330+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:31.528330+0000","last_peered":"2024-08-20T06:36:31.528330+0000","last_clean":"2024-08-20T06:36:31.528330+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:31.528330+0000","last_undegraded":"2024-08-20T06:36:31.528330+0000","last_fullsized":"2024-08-20T06:36:31.528330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529088+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:31.529088+0000","last_peered":"2024-08-20T06:36:31.529088+0000","last_clean":"2024-08-20T06:36:31.529088+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:31.529088+0000","last_undegraded":"2024-08-20T06:36:31.529088+0000","last_fullsized":"2024-08-20T06:36:31.529088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'6","reported_seq":138,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:54.144297+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:54.144297+0000","last_peered":"2024-08-20T06:36:54.144297+0000","last_clean":"2024-08-20T06:36:54.144297+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:54.144297+0000","last_undegraded":"2024-08-20T06:36:54.144297+0000","last_fullsized":"2024-08-20T06:36:54.144297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541085+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:31.541085+0000","last_peered":"2024-08-20T06:36:31.541085+0000","last_clean":"2024-08-20T06:36:31.541085+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:31.541085+0000","last_undegraded":"2024-08-20T06:36:31.541085+0000","last_fullsized":"2024-08-20T06:36:31.541085+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226778+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:03.226778+0000","last_peered":"2024-08-20T06:36:03.226778+0000","last_clean":"2024-08-20T06:36:03.226778+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:03.226778+0000","last_undegraded":"2024-08-20T06:36:03.226778+0000","last_fullsized":"2024-08-20T06:36:03.226778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523940+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:31.523940+0000","last_peered":"2024-08-20T06:36:31.523940+0000","last_clean":"2024-08-20T06:36:31.523940+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:31.523940+0000","last_undegraded":"2024-08-20T06:36:31.523940+0000","last_fullsized":"2024-08-20T06:36:31.523940+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534641+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:31.534641+0000","last_peered":"2024-08-20T06:36:31.534641+0000","last_clean":"2024-08-20T06:36:31.534641+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:31.534641+0000","last_undegraded":"2024-08-20T06:36:31.534641+0000","last_fullsized":"2024-08-20T06:36:31.534641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'4","reported_seq":135,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.147134+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:56.147134+0000","last_peered":"2024-08-20T06:36:56.147134+0000","last_clean":"2024-08-20T06:36:56.147134+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:56.147134+0000","last_undegraded":"2024-08-20T06:36:56.147134+0000","last_fullsized":"2024-08-20T06:36:56.147134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541403+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:31.541403+0000","last_peered":"2024-08-20T06:36:31.541403+0000","last_clean":"2024-08-20T06:36:31.541403+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:31.541403+0000","last_undegraded":"2024-08-20T06:36:31.541403+0000","last_fullsized":"2024-08-20T06:36:31.541403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"73'9","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531015+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:31.531015+0000","last_peered":"2024-08-20T06:36:31.531015+0000","last_clean":"2024-08-20T06:36:31.531015+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:31.531015+0000","last_undegraded":"2024-08-20T06:36:31.531015+0000","last_fullsized":"2024-08-20T06:36:31.531015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542985+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:31.542985+0000","last_peered":"2024-08-20T06:36:31.542985+0000","last_clean":"2024-08-20T06:36:31.542985+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:31.542985+0000","last_undegraded":"2024-08-20T06:36:31.542985+0000","last_fullsized":"2024-08-20T06:36:31.542985+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"73'35","reported_seq":301,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.428964+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:56.428964+0000","last_peered":"2024-08-20T06:36:56.428964+0000","last_clean":"2024-08-20T06:36:56.428964+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:56.428964+0000","last_undegraded":"2024-08-20T06:36:56.428964+0000","last_fullsized":"2024-08-20T06:36:56.428964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67197,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531845+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:31.531845+0000","last_peered":"2024-08-20T06:36:31.531845+0000","last_clean":"2024-08-20T06:36:31.531845+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:31.531845+0000","last_undegraded":"2024-08-20T06:36:31.531845+0000","last_fullsized":"2024-08-20T06:36:31.531845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532459+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:31.532459+0000","last_peered":"2024-08-20T06:36:31.532459+0000","last_clean":"2024-08-20T06:36:31.532459+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:31.532459+0000","last_undegraded":"2024-08-20T06:36:31.532459+0000","last_fullsized":"2024-08-20T06:36:31.532459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"73'15","reported_seq":40,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534636+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:31.534636+0000","last_peered":"2024-08-20T06:36:31.534636+0000","last_clean":"2024-08-20T06:36:31.534636+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:31.534636+0000","last_undegraded":"2024-08-20T06:36:31.534636+0000","last_fullsized":"2024-08-20T06:36:31.534636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'5","reported_seq":140,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:57.159656+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:57.159656+0000","last_peered":"2024-08-20T06:36:57.159656+0000","last_clean":"2024-08-20T06:36:57.159656+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:57.159656+0000","last_undegraded":"2024-08-20T06:36:57.159656+0000","last_fullsized":"2024-08-20T06:36:57.159656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532941+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:31.532941+0000","last_peered":"2024-08-20T06:36:31.532941+0000","last_clean":"2024-08-20T06:36:31.532941+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:31.532941+0000","last_undegraded":"2024-08-20T06:36:31.532941+0000","last_fullsized":"2024-08-20T06:36:31.532941+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542040+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:31.542040+0000","last_peered":"2024-08-20T06:36:31.542040+0000","last_clean":"2024-08-20T06:36:31.542040+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:31.542040+0000","last_undegraded":"2024-08-20T06:36:31.542040+0000","last_fullsized":"2024-08-20T06:36:31.542040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527503+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:31.527503+0000","last_peered":"2024-08-20T06:36:31.527503+0000","last_clean":"2024-08-20T06:36:31.527503+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:31.527503+0000","last_undegraded":"2024-08-20T06:36:31.527503+0000","last_fullsized":"2024-08-20T06:36:31.527503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227139+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:03.227139+0000","last_peered":"2024-08-20T06:36:03.227139+0000","last_clean":"2024-08-20T06:36:03.227139+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:03.227139+0000","last_undegraded":"2024-08-20T06:36:03.227139+0000","last_fullsized":"2024-08-20T06:36:03.227139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530907+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:31.530907+0000","last_peered":"2024-08-20T06:36:31.530907+0000","last_clean":"2024-08-20T06:36:31.530907+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:31.530907+0000","last_undegraded":"2024-08-20T06:36:31.530907+0000","last_fullsized":"2024-08-20T06:36:31.530907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'7","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531455+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:31.531455+0000","last_peered":"2024-08-20T06:36:31.531455+0000","last_clean":"2024-08-20T06:36:31.531455+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:31.531455+0000","last_undegraded":"2024-08-20T06:36:31.531455+0000","last_fullsized":"2024-08-20T06:36:31.531455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538900+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:31.538900+0000","last_peered":"2024-08-20T06:36:31.538900+0000","last_clean":"2024-08-20T06:36:31.538900+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:31.538900+0000","last_undegraded":"2024-08-20T06:36:31.538900+0000","last_fullsized":"2024-08-20T06:36:31.538900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":92,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122684+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122684+0000","last_peered":"2024-08-20T06:36:03.122684+0000","last_clean":"2024-08-20T06:36:03.122684+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122684+0000","last_undegraded":"2024-08-20T06:36:03.122684+0000","last_fullsized":"2024-08-20T06:36:03.122684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339216+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:03.339216+0000","last_peered":"2024-08-20T06:36:03.339216+0000","last_clean":"2024-08-20T06:36:03.339216+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:03.339216+0000","last_undegraded":"2024-08-20T06:36:03.339216+0000","last_fullsized":"2024-08-20T06:36:03.339216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529343+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:31.529343+0000","last_peered":"2024-08-20T06:36:31.529343+0000","last_clean":"2024-08-20T06:36:31.529343+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:31.529343+0000","last_undegraded":"2024-08-20T06:36:31.529343+0000","last_fullsized":"2024-08-20T06:36:31.529343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.135010+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.135010+0000","last_peered":"2024-08-20T06:36:03.135010+0000","last_clean":"2024-08-20T06:36:03.135010+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.135010+0000","last_undegraded":"2024-08-20T06:36:03.135010+0000","last_fullsized":"2024-08-20T06:36:03.135010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525687+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:31.525687+0000","last_peered":"2024-08-20T06:36:31.525687+0000","last_clean":"2024-08-20T06:36:31.525687+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:31.525687+0000","last_undegraded":"2024-08-20T06:36:31.525687+0000","last_fullsized":"2024-08-20T06:36:31.525687+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'5","reported_seq":136,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.157371+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:56.157371+0000","last_peered":"2024-08-20T06:36:56.157371+0000","last_clean":"2024-08-20T06:36:56.157371+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:56.157371+0000","last_undegraded":"2024-08-20T06:36:56.157371+0000","last_fullsized":"2024-08-20T06:36:56.157371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541026+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:31.541026+0000","last_peered":"2024-08-20T06:36:31.541026+0000","last_clean":"2024-08-20T06:36:31.541026+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:31.541026+0000","last_undegraded":"2024-08-20T06:36:31.541026+0000","last_fullsized":"2024-08-20T06:36:31.541026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530300+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:31.530300+0000","last_peered":"2024-08-20T06:36:31.530300+0000","last_clean":"2024-08-20T06:36:31.530300+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:31.530300+0000","last_undegraded":"2024-08-20T06:36:31.530300+0000","last_fullsized":"2024-08-20T06:36:31.530300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537954+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:31.537954+0000","last_peered":"2024-08-20T06:36:31.537954+0000","last_clean":"2024-08-20T06:36:31.537954+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:31.537954+0000","last_undegraded":"2024-08-20T06:36:31.537954+0000","last_fullsized":"2024-08-20T06:36:31.537954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526170+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:31.526170+0000","last_peered":"2024-08-20T06:36:31.526170+0000","last_clean":"2024-08-20T06:36:31.526170+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:31.526170+0000","last_undegraded":"2024-08-20T06:36:31.526170+0000","last_fullsized":"2024-08-20T06:36:31.526170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'11","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123684+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123684+0000","last_peered":"2024-08-20T06:36:03.123684+0000","last_clean":"2024-08-20T06:36:03.123684+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123684+0000","last_undegraded":"2024-08-20T06:36:03.123684+0000","last_fullsized":"2024-08-20T06:36:03.123684+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525153+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:31.525153+0000","last_peered":"2024-08-20T06:36:31.525153+0000","last_clean":"2024-08-20T06:36:31.525153+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:31.525153+0000","last_undegraded":"2024-08-20T06:36:31.525153+0000","last_fullsized":"2024-08-20T06:36:31.525153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339652+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:03.339652+0000","last_peered":"2024-08-20T06:36:03.339652+0000","last_clean":"2024-08-20T06:36:03.339652+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:03.339652+0000","last_undegraded":"2024-08-20T06:36:03.339652+0000","last_fullsized":"2024-08-20T06:36:03.339652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540510+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:31.540510+0000","last_peered":"2024-08-20T06:36:31.540510+0000","last_clean":"2024-08-20T06:36:31.540510+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:31.540510+0000","last_undegraded":"2024-08-20T06:36:31.540510+0000","last_fullsized":"2024-08-20T06:36:31.540510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.493499+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:31.493499+0000","last_peered":"2024-08-20T06:36:31.493499+0000","last_clean":"2024-08-20T06:36:31.493499+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:31.493499+0000","last_undegraded":"2024-08-20T06:36:31.493499+0000","last_fullsized":"2024-08-20T06:36:31.493499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124496+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:03.124496+0000","last_peered":"2024-08-20T06:36:03.124496+0000","last_clean":"2024-08-20T06:36:03.124496+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:03.124496+0000","last_undegraded":"2024-08-20T06:36:03.124496+0000","last_fullsized":"2024-08-20T06:36:03.124496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527640+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:31.527640+0000","last_peered":"2024-08-20T06:36:31.527640+0000","last_clean":"2024-08-20T06:36:31.527640+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:31.527640+0000","last_undegraded":"2024-08-20T06:36:31.527640+0000","last_fullsized":"2024-08-20T06:36:31.527640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535568+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:31.535568+0000","last_peered":"2024-08-20T06:36:31.535568+0000","last_clean":"2024-08-20T06:36:31.535568+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:31.535568+0000","last_undegraded":"2024-08-20T06:36:31.535568+0000","last_fullsized":"2024-08-20T06:36:31.535568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.539026+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:31.539026+0000","last_peered":"2024-08-20T06:36:31.539026+0000","last_clean":"2024-08-20T06:36:31.539026+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:31.539026+0000","last_undegraded":"2024-08-20T06:36:31.539026+0000","last_fullsized":"2024-08-20T06:36:31.539026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542524+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:31.542524+0000","last_peered":"2024-08-20T06:36:31.542524+0000","last_clean":"2024-08-20T06:36:31.542524+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:31.542524+0000","last_undegraded":"2024-08-20T06:36:31.542524+0000","last_fullsized":"2024-08-20T06:36:31.542524+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541846+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:31.541846+0000","last_peered":"2024-08-20T06:36:31.541846+0000","last_clean":"2024-08-20T06:36:31.541846+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:31.541846+0000","last_undegraded":"2024-08-20T06:36:31.541846+0000","last_fullsized":"2024-08-20T06:36:31.541846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541363+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:31.541363+0000","last_peered":"2024-08-20T06:36:31.541363+0000","last_clean":"2024-08-20T06:36:31.541363+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:31.541363+0000","last_undegraded":"2024-08-20T06:36:31.541363+0000","last_fullsized":"2024-08-20T06:36:31.541363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125324+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:03.125324+0000","last_peered":"2024-08-20T06:36:03.125324+0000","last_clean":"2024-08-20T06:36:03.125324+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:03.125324+0000","last_undegraded":"2024-08-20T06:36:03.125324+0000","last_fullsized":"2024-08-20T06:36:03.125324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528633+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:31.528633+0000","last_peered":"2024-08-20T06:36:31.528633+0000","last_clean":"2024-08-20T06:36:31.528633+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:31.528633+0000","last_undegraded":"2024-08-20T06:36:31.528633+0000","last_fullsized":"2024-08-20T06:36:31.528633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72184,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":672,"num_read_kb":408,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037801,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1045241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168622,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332148,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528376,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1409024,"data_stored":978360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":802816,"data_stored":385304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1421312,"data_stored":977551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051655,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":453822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607634,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":447589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":386575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:37:00.263 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-08-20T06:37:00.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/2351665167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 msgr2=0x7f9e7410ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/2351665167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e7410ab30 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c009cd0 tx=0x7f9e6c02a8e0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/2351665167 shutdown_connections 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/2351665167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e7410ab30 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/2351665167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741037f0 0x7f9e74103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/2351665167 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741026a0 0x7f9e74102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/2351665167 >> 172.21.15.107:0/2351665167 conn(0x7f9e740fdf40 msgr2=0x7f9e74100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/2351665167 shutdown_connections 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/2351665167 wait complete. 2024-08-20T06:37:00.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 Processor -- start 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.397+0000 7f9e7aa5f640 1 -- start start 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 0x7f9e74198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741037f0 0x7f9e74198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7410e210 con 0x7f9e741041b0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7410e090 con 0x7f9e741037f0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7410e390 con 0x7f9e741026a0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60238/0 (socket says 172.21.15.107:60238) 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 -- 172.21.15.107:0/4019642262 learned_addr learned my addr 172.21.15.107:0/4019642262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e737fe640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741037f0 0x7f9e74198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e73fff640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 0x7f9e74198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.401 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 -- 172.21.15.107:0/4019642262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 msgr2=0x7f9e74198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 0x7f9e74198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 -- 172.21.15.107:0/4019642262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741037f0 msgr2=0x7f9e74198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741037f0 0x7f9e74198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 -- 172.21.15.107:0/4019642262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7419a4b0 con 0x7f9e741041b0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e73fff640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 0x7f9e74198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e78ffc640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c003ba0 tx=0x7f9e6c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c004240 con 0x7f9e741041b0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9e6c0305a0 con 0x7f9e741041b0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e6c0466e0 con 0x7f9e741041b0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7419a740 con 0x7f9e741041b0 2024-08-20T06:37:00.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.398+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e741a0e70 con 0x7f9e741041b0 2024-08-20T06:37:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.399+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e74102aa0 con 0x7f9e741041b0 2024-08-20T06:37:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.401+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f9e6c030a50 con 0x7f9e741041b0 2024-08-20T06:37:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.401+0000 7f9e717fa640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 0x7f9e5c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.401+0000 7f9e73fff640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 0x7f9e5c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:00.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.401+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9e6c02e030 con 0x7f9e741041b0 2024-08-20T06:37:00.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.402+0000 7f9e73fff640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 0x7f9e5c0b6120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e64005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:00.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.404+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f9e6c04b060 con 0x7f9e741041b0 2024-08-20T06:37:00.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.575+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 --> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9e740632c0 con 0x7f9e5c0b3c60 2024-08-20T06:37:00.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.579+0000 7f9e717fa640 1 -- 172.21.15.107:0/4019642262 <== mgr.14150 v2:172.21.15.107:6800/1083091367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258845 (secure 0 0 0) 0x7f9e740632c0 con 0x7f9e5c0b3c60 2024-08-20T06:37:00.583 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:37:00.585 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-08-20T06:37:00.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 msgr2=0x7f9e5c0b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 0x7f9e5c0b6120 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9e640058c0 tx=0x7f9e64005d30 comp rx=0 tx=0).stop 2024-08-20T06:37:00.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 msgr2=0x7f9e74199bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9e6c003ba0 tx=0x7f9e6c003bd0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 shutdown_connections 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9e741041b0 0x7f9e74199bf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9e741037f0 0x7f9e74198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9e5c0b3c60 0x7f9e5c0b6120 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 --2- 172.21.15.107:0/4019642262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e741026a0 0x7f9e74198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 >> 172.21.15.107:0/4019642262 conn(0x7f9e740fdf40 msgr2=0x7f9e74106b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 shutdown_connections 2024-08-20T06:37:00.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:00.582+0000 7f9e7aa5f640 1 -- 172.21.15.107:0/4019642262 wait complete. 2024-08-20T06:37:00.594 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-08-20T06:36:59.334749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662786,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":472,"num_write":299,"num_write_kb":2276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":3013763,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369072,"kb_used_data":11504,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572752,"statfs":{"total":1151940427776,"available":1151562498048,"internally_reserved":0,"allocated":11780096,"data_stored":6603933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6575,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":74,"num_read_kb":39,"num_write":8,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"3.3f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.524915+0000","last_change":"2024-08-20T06:34:59.942304+0000","last_active":"2024-08-20T06:36:31.524915+0000","last_peered":"2024-08-20T06:36:31.524915+0000","last_clean":"2024-08-20T06:36:31.524915+0000","last_became_active":"2024-08-20T06:34:59.941829+0000","last_became_peered":"2024-08-20T06:34:59.941829+0000","last_unstale":"2024-08-20T06:36:31.524915+0000","last_undegraded":"2024-08-20T06:36:31.524915+0000","last_fullsized":"2024-08-20T06:36:31.524915+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:17:47.816544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534240+0000","last_change":"2024-08-20T06:34:59.933888+0000","last_active":"2024-08-20T06:36:31.534240+0000","last_peered":"2024-08-20T06:36:31.534240+0000","last_clean":"2024-08-20T06:36:31.534240+0000","last_became_active":"2024-08-20T06:34:59.933594+0000","last_became_peered":"2024-08-20T06:34:59.933594+0000","last_unstale":"2024-08-20T06:36:31.534240+0000","last_undegraded":"2024-08-20T06:36:31.534240+0000","last_fullsized":"2024-08-20T06:36:31.534240+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:42:33.198460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540252+0000","last_change":"2024-08-20T06:34:59.932947+0000","last_active":"2024-08-20T06:36:31.540252+0000","last_peered":"2024-08-20T06:36:31.540252+0000","last_clean":"2024-08-20T06:36:31.540252+0000","last_became_active":"2024-08-20T06:34:59.932521+0000","last_became_peered":"2024-08-20T06:34:59.932521+0000","last_unstale":"2024-08-20T06:36:31.540252+0000","last_undegraded":"2024-08-20T06:36:31.540252+0000","last_fullsized":"2024-08-20T06:36:31.540252+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:45:45.539313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.515382+0000","last_change":"2024-08-20T06:36:03.126923+0000","last_active":"2024-08-20T06:36:31.515382+0000","last_peered":"2024-08-20T06:36:31.515382+0000","last_clean":"2024-08-20T06:36:31.515382+0000","last_became_active":"2024-08-20T06:36:03.126571+0000","last_became_peered":"2024-08-20T06:36:03.126571+0000","last_unstale":"2024-08-20T06:36:31.515382+0000","last_undegraded":"2024-08-20T06:36:31.515382+0000","last_fullsized":"2024-08-20T06:36:31.515382+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:05.974173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.514978+0000","last_change":"2024-08-20T06:34:59.933376+0000","last_active":"2024-08-20T06:36:31.514978+0000","last_peered":"2024-08-20T06:36:31.514978+0000","last_clean":"2024-08-20T06:36:31.514978+0000","last_became_active":"2024-08-20T06:34:59.931837+0000","last_became_peered":"2024-08-20T06:34:59.931837+0000","last_unstale":"2024-08-20T06:36:31.514978+0000","last_undegraded":"2024-08-20T06:36:31.514978+0000","last_fullsized":"2024-08-20T06:36:31.514978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:58:26.937799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537525+0000","last_change":"2024-08-20T06:34:59.930557+0000","last_active":"2024-08-20T06:36:31.537525+0000","last_peered":"2024-08-20T06:36:31.537525+0000","last_clean":"2024-08-20T06:36:31.537525+0000","last_became_active":"2024-08-20T06:34:59.930216+0000","last_became_peered":"2024-08-20T06:34:59.930216+0000","last_unstale":"2024-08-20T06:36:31.537525+0000","last_undegraded":"2024-08-20T06:36:31.537525+0000","last_fullsized":"2024-08-20T06:36:31.537525+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:25:41.071728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"73'5","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528038+0000","last_change":"2024-08-20T06:36:03.126856+0000","last_active":"2024-08-20T06:36:31.528038+0000","last_peered":"2024-08-20T06:36:31.528038+0000","last_clean":"2024-08-20T06:36:31.528038+0000","last_became_active":"2024-08-20T06:36:03.126524+0000","last_became_peered":"2024-08-20T06:36:03.126524+0000","last_unstale":"2024-08-20T06:36:31.528038+0000","last_undegraded":"2024-08-20T06:36:31.528038+0000","last_fullsized":"2024-08-20T06:36:31.528038+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:57.471447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,8],"acting":[7,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535771+0000","last_change":"2024-08-20T06:34:59.936986+0000","last_active":"2024-08-20T06:36:31.535771+0000","last_peered":"2024-08-20T06:36:31.535771+0000","last_clean":"2024-08-20T06:36:31.535771+0000","last_became_active":"2024-08-20T06:34:59.936707+0000","last_became_peered":"2024-08-20T06:34:59.936707+0000","last_unstale":"2024-08-20T06:36:31.535771+0000","last_undegraded":"2024-08-20T06:36:31.535771+0000","last_fullsized":"2024-08-20T06:36:31.535771+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:25.670931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.543127+0000","last_change":"2024-08-20T06:34:59.939018+0000","last_active":"2024-08-20T06:36:31.543127+0000","last_peered":"2024-08-20T06:36:31.543127+0000","last_clean":"2024-08-20T06:36:31.543127+0000","last_became_active":"2024-08-20T06:34:59.938641+0000","last_became_peered":"2024-08-20T06:34:59.938641+0000","last_unstale":"2024-08-20T06:36:31.543127+0000","last_undegraded":"2024-08-20T06:36:31.543127+0000","last_fullsized":"2024-08-20T06:36:31.543127+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:37:05.431865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"73'16","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.536670+0000","last_change":"2024-08-20T06:34:59.924326+0000","last_active":"2024-08-20T06:36:31.536670+0000","last_peered":"2024-08-20T06:36:31.536670+0000","last_clean":"2024-08-20T06:36:31.536670+0000","last_became_active":"2024-08-20T06:34:59.924015+0000","last_became_peered":"2024-08-20T06:34:59.924015+0000","last_unstale":"2024-08-20T06:36:31.536670+0000","last_undegraded":"2024-08-20T06:36:31.536670+0000","last_fullsized":"2024-08-20T06:36:31.536670+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:16:44.314153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538446+0000","last_change":"2024-08-20T06:34:59.941079+0000","last_active":"2024-08-20T06:36:31.538446+0000","last_peered":"2024-08-20T06:36:31.538446+0000","last_clean":"2024-08-20T06:36:31.538446+0000","last_became_active":"2024-08-20T06:34:59.940742+0000","last_became_peered":"2024-08-20T06:34:59.940742+0000","last_unstale":"2024-08-20T06:36:31.538446+0000","last_undegraded":"2024-08-20T06:36:31.538446+0000","last_fullsized":"2024-08-20T06:36:31.538446+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:02:33.372609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541569+0000","last_change":"2024-08-20T06:34:59.933167+0000","last_active":"2024-08-20T06:36:31.541569+0000","last_peered":"2024-08-20T06:36:31.541569+0000","last_clean":"2024-08-20T06:36:31.541569+0000","last_became_active":"2024-08-20T06:34:59.932665+0000","last_became_peered":"2024-08-20T06:34:59.932665+0000","last_unstale":"2024-08-20T06:36:31.541569+0000","last_undegraded":"2024-08-20T06:36:31.541569+0000","last_fullsized":"2024-08-20T06:36:31.541569+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:30:59.615972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530179+0000","last_change":"2024-08-20T06:34:59.932518+0000","last_active":"2024-08-20T06:36:31.530179+0000","last_peered":"2024-08-20T06:36:31.530179+0000","last_clean":"2024-08-20T06:36:31.530179+0000","last_became_active":"2024-08-20T06:34:59.932050+0000","last_became_peered":"2024-08-20T06:34:59.932050+0000","last_unstale":"2024-08-20T06:36:31.530179+0000","last_undegraded":"2024-08-20T06:36:31.530179+0000","last_fullsized":"2024-08-20T06:36:31.530179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:00:50.402003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534684+0000","last_change":"2024-08-20T06:34:59.931886+0000","last_active":"2024-08-20T06:36:31.534684+0000","last_peered":"2024-08-20T06:36:31.534684+0000","last_clean":"2024-08-20T06:36:31.534684+0000","last_became_active":"2024-08-20T06:34:59.931586+0000","last_became_peered":"2024-08-20T06:34:59.931586+0000","last_unstale":"2024-08-20T06:36:31.534684+0000","last_undegraded":"2024-08-20T06:36:31.534684+0000","last_fullsized":"2024-08-20T06:36:31.534684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.777846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540482+0000","last_change":"2024-08-20T06:34:59.929104+0000","last_active":"2024-08-20T06:36:31.540482+0000","last_peered":"2024-08-20T06:36:31.540482+0000","last_clean":"2024-08-20T06:36:31.540482+0000","last_became_active":"2024-08-20T06:34:59.928804+0000","last_became_peered":"2024-08-20T06:34:59.928804+0000","last_unstale":"2024-08-20T06:36:31.540482+0000","last_undegraded":"2024-08-20T06:36:31.540482+0000","last_fullsized":"2024-08-20T06:36:31.540482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:01:50.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542459+0000","last_change":"2024-08-20T06:34:59.935731+0000","last_active":"2024-08-20T06:36:31.542459+0000","last_peered":"2024-08-20T06:36:31.542459+0000","last_clean":"2024-08-20T06:36:31.542459+0000","last_became_active":"2024-08-20T06:34:59.935427+0000","last_became_peered":"2024-08-20T06:34:59.935427+0000","last_unstale":"2024-08-20T06:36:31.542459+0000","last_undegraded":"2024-08-20T06:36:31.542459+0000","last_fullsized":"2024-08-20T06:36:31.542459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:55:59.065951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525439+0000","last_change":"2024-08-20T06:34:59.941049+0000","last_active":"2024-08-20T06:36:31.525439+0000","last_peered":"2024-08-20T06:36:31.525439+0000","last_clean":"2024-08-20T06:36:31.525439+0000","last_became_active":"2024-08-20T06:34:59.939483+0000","last_became_peered":"2024-08-20T06:34:59.939483+0000","last_unstale":"2024-08-20T06:36:31.525439+0000","last_undegraded":"2024-08-20T06:36:31.525439+0000","last_fullsized":"2024-08-20T06:36:31.525439+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:24:00.476722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535381+0000","last_change":"2024-08-20T06:34:59.932442+0000","last_active":"2024-08-20T06:36:31.535381+0000","last_peered":"2024-08-20T06:36:31.535381+0000","last_clean":"2024-08-20T06:36:31.535381+0000","last_became_active":"2024-08-20T06:34:59.932149+0000","last_became_peered":"2024-08-20T06:34:59.932149+0000","last_unstale":"2024-08-20T06:36:31.535381+0000","last_undegraded":"2024-08-20T06:36:31.535381+0000","last_fullsized":"2024-08-20T06:36:31.535381+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:58:55.670965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526562+0000","last_change":"2024-08-20T06:34:59.932331+0000","last_active":"2024-08-20T06:36:31.526562+0000","last_peered":"2024-08-20T06:36:31.526562+0000","last_clean":"2024-08-20T06:36:31.526562+0000","last_became_active":"2024-08-20T06:34:59.931858+0000","last_became_peered":"2024-08-20T06:34:59.931858+0000","last_unstale":"2024-08-20T06:36:31.526562+0000","last_undegraded":"2024-08-20T06:36:31.526562+0000","last_fullsized":"2024-08-20T06:36:31.526562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:04:50.899983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527011+0000","last_change":"2024-08-20T06:34:59.933623+0000","last_active":"2024-08-20T06:36:31.527011+0000","last_peered":"2024-08-20T06:36:31.527011+0000","last_clean":"2024-08-20T06:36:31.527011+0000","last_became_active":"2024-08-20T06:34:59.933288+0000","last_became_peered":"2024-08-20T06:34:59.933288+0000","last_unstale":"2024-08-20T06:36:31.527011+0000","last_undegraded":"2024-08-20T06:36:31.527011+0000","last_fullsized":"2024-08-20T06:36:31.527011+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:39:50.763163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.533876+0000","last_change":"2024-08-20T06:34:59.930123+0000","last_active":"2024-08-20T06:36:31.533876+0000","last_peered":"2024-08-20T06:36:31.533876+0000","last_clean":"2024-08-20T06:36:31.533876+0000","last_became_active":"2024-08-20T06:34:59.929798+0000","last_became_peered":"2024-08-20T06:34:59.929798+0000","last_unstale":"2024-08-20T06:36:31.533876+0000","last_undegraded":"2024-08-20T06:36:31.533876+0000","last_fullsized":"2024-08-20T06:36:31.533876+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:49:52.777034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542900+0000","last_change":"2024-08-20T06:34:59.935576+0000","last_active":"2024-08-20T06:36:31.542900+0000","last_peered":"2024-08-20T06:36:31.542900+0000","last_clean":"2024-08-20T06:36:31.542900+0000","last_became_active":"2024-08-20T06:34:59.934972+0000","last_became_peered":"2024-08-20T06:34:59.934972+0000","last_unstale":"2024-08-20T06:36:31.542900+0000","last_undegraded":"2024-08-20T06:36:31.542900+0000","last_fullsized":"2024-08-20T06:36:31.542900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:10.280088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523834+0000","last_change":"2024-08-20T06:34:59.931104+0000","last_active":"2024-08-20T06:36:31.523834+0000","last_peered":"2024-08-20T06:36:31.523834+0000","last_clean":"2024-08-20T06:36:31.523834+0000","last_became_active":"2024-08-20T06:34:59.930725+0000","last_became_peered":"2024-08-20T06:34:59.930725+0000","last_unstale":"2024-08-20T06:36:31.523834+0000","last_undegraded":"2024-08-20T06:36:31.523834+0000","last_fullsized":"2024-08-20T06:36:31.523834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:46:40.086728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529739+0000","last_change":"2024-08-20T06:34:59.931552+0000","last_active":"2024-08-20T06:36:31.529739+0000","last_peered":"2024-08-20T06:36:31.529739+0000","last_clean":"2024-08-20T06:36:31.529739+0000","last_became_active":"2024-08-20T06:34:59.931221+0000","last_became_peered":"2024-08-20T06:34:59.931221+0000","last_unstale":"2024-08-20T06:36:31.529739+0000","last_undegraded":"2024-08-20T06:36:31.529739+0000","last_fullsized":"2024-08-20T06:36:31.529739+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:09:34.650075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.521121+0000","last_change":"2024-08-20T06:34:59.936179+0000","last_active":"2024-08-20T06:36:31.521121+0000","last_peered":"2024-08-20T06:36:31.521121+0000","last_clean":"2024-08-20T06:36:31.521121+0000","last_became_active":"2024-08-20T06:34:59.935859+0000","last_became_peered":"2024-08-20T06:34:59.935859+0000","last_unstale":"2024-08-20T06:36:31.521121+0000","last_undegraded":"2024-08-20T06:36:31.521121+0000","last_fullsized":"2024-08-20T06:36:31.521121+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:12:25.096307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529168+0000","last_change":"2024-08-20T06:34:59.936301+0000","last_active":"2024-08-20T06:36:31.529168+0000","last_peered":"2024-08-20T06:36:31.529168+0000","last_clean":"2024-08-20T06:36:31.529168+0000","last_became_active":"2024-08-20T06:34:59.936006+0000","last_became_peered":"2024-08-20T06:34:59.936006+0000","last_unstale":"2024-08-20T06:36:31.529168+0000","last_undegraded":"2024-08-20T06:36:31.529168+0000","last_fullsized":"2024-08-20T06:36:31.529168+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:17:19.440527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526180+0000","last_change":"2024-08-20T06:34:59.938127+0000","last_active":"2024-08-20T06:36:31.526180+0000","last_peered":"2024-08-20T06:36:31.526180+0000","last_clean":"2024-08-20T06:36:31.526180+0000","last_became_active":"2024-08-20T06:34:59.937819+0000","last_became_peered":"2024-08-20T06:34:59.937819+0000","last_unstale":"2024-08-20T06:36:31.526180+0000","last_undegraded":"2024-08-20T06:36:31.526180+0000","last_fullsized":"2024-08-20T06:36:31.526180+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:41:54.197440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"73'4","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.522146+0000","last_change":"2024-08-20T06:34:59.930147+0000","last_active":"2024-08-20T06:36:31.522146+0000","last_peered":"2024-08-20T06:36:31.522146+0000","last_clean":"2024-08-20T06:36:31.522146+0000","last_became_active":"2024-08-20T06:34:59.929826+0000","last_became_peered":"2024-08-20T06:34:59.929826+0000","last_unstale":"2024-08-20T06:36:31.522146+0000","last_undegraded":"2024-08-20T06:36:31.522146+0000","last_fullsized":"2024-08-20T06:36:31.522146+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:05:25.091410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540463+0000","last_change":"2024-08-20T06:34:59.932246+0000","last_active":"2024-08-20T06:36:31.540463+0000","last_peered":"2024-08-20T06:36:31.540463+0000","last_clean":"2024-08-20T06:36:31.540463+0000","last_became_active":"2024-08-20T06:34:59.931287+0000","last_became_peered":"2024-08-20T06:34:59.931287+0000","last_unstale":"2024-08-20T06:36:31.540463+0000","last_undegraded":"2024-08-20T06:36:31.540463+0000","last_fullsized":"2024-08-20T06:36:31.540463+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:14.639174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.520194+0000","last_change":"2024-08-20T06:34:59.933675+0000","last_active":"2024-08-20T06:36:31.520194+0000","last_peered":"2024-08-20T06:36:31.520194+0000","last_clean":"2024-08-20T06:36:31.520194+0000","last_became_active":"2024-08-20T06:34:59.932589+0000","last_became_peered":"2024-08-20T06:34:59.932589+0000","last_unstale":"2024-08-20T06:36:31.520194+0000","last_undegraded":"2024-08-20T06:36:31.520194+0000","last_fullsized":"2024-08-20T06:36:31.520194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:41:47.741765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"73'12","reported_seq":30,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530539+0000","last_change":"2024-08-20T06:34:59.930273+0000","last_active":"2024-08-20T06:36:31.530539+0000","last_peered":"2024-08-20T06:36:31.530539+0000","last_clean":"2024-08-20T06:36:31.530539+0000","last_became_active":"2024-08-20T06:34:59.929955+0000","last_became_peered":"2024-08-20T06:34:59.929955+0000","last_unstale":"2024-08-20T06:36:31.530539+0000","last_undegraded":"2024-08-20T06:36:31.530539+0000","last_fullsized":"2024-08-20T06:36:31.530539+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:50:24.284553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528330+0000","last_change":"2024-08-20T06:34:59.940441+0000","last_active":"2024-08-20T06:36:31.528330+0000","last_peered":"2024-08-20T06:36:31.528330+0000","last_clean":"2024-08-20T06:36:31.528330+0000","last_became_active":"2024-08-20T06:34:59.940088+0000","last_became_peered":"2024-08-20T06:34:59.940088+0000","last_unstale":"2024-08-20T06:36:31.528330+0000","last_undegraded":"2024-08-20T06:36:31.528330+0000","last_fullsized":"2024-08-20T06:36:31.528330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:38:23.956517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529088+0000","last_change":"2024-08-20T06:34:59.933966+0000","last_active":"2024-08-20T06:36:31.529088+0000","last_peered":"2024-08-20T06:36:31.529088+0000","last_clean":"2024-08-20T06:36:31.529088+0000","last_became_active":"2024-08-20T06:34:59.933639+0000","last_became_peered":"2024-08-20T06:34:59.933639+0000","last_unstale":"2024-08-20T06:36:31.529088+0000","last_undegraded":"2024-08-20T06:36:31.529088+0000","last_fullsized":"2024-08-20T06:36:31.529088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:21.079688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"73'6","reported_seq":138,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:54.144297+0000","last_change":"2024-08-20T06:34:57.920710+0000","last_active":"2024-08-20T06:36:54.144297+0000","last_peered":"2024-08-20T06:36:54.144297+0000","last_clean":"2024-08-20T06:36:54.144297+0000","last_became_active":"2024-08-20T06:34:57.920263+0000","last_became_peered":"2024-08-20T06:34:57.920263+0000","last_unstale":"2024-08-20T06:36:54.144297+0000","last_undegraded":"2024-08-20T06:36:54.144297+0000","last_fullsized":"2024-08-20T06:36:54.144297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:47:09.891792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1115,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":109,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541085+0000","last_change":"2024-08-20T06:34:59.931342+0000","last_active":"2024-08-20T06:36:31.541085+0000","last_peered":"2024-08-20T06:36:31.541085+0000","last_clean":"2024-08-20T06:36:31.541085+0000","last_became_active":"2024-08-20T06:34:59.930860+0000","last_became_peered":"2024-08-20T06:34:59.930860+0000","last_unstale":"2024-08-20T06:36:31.541085+0000","last_undegraded":"2024-08-20T06:36:31.541085+0000","last_fullsized":"2024-08-20T06:36:31.541085+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:17:11.922404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124811+0000","last_change":"2024-08-20T06:34:57.913272+0000","last_active":"2024-08-20T06:36:03.124811+0000","last_peered":"2024-08-20T06:36:03.124811+0000","last_clean":"2024-08-20T06:36:03.124811+0000","last_became_active":"2024-08-20T06:34:57.912913+0000","last_became_peered":"2024-08-20T06:34:57.912913+0000","last_unstale":"2024-08-20T06:36:03.124811+0000","last_undegraded":"2024-08-20T06:36:03.124811+0000","last_fullsized":"2024-08-20T06:36:03.124811+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:57:43.039374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.226778+0000","last_change":"2024-08-20T06:34:57.913199+0000","last_active":"2024-08-20T06:36:03.226778+0000","last_peered":"2024-08-20T06:36:03.226778+0000","last_clean":"2024-08-20T06:36:03.226778+0000","last_became_active":"2024-08-20T06:34:57.912870+0000","last_became_peered":"2024-08-20T06:34:57.912870+0000","last_unstale":"2024-08-20T06:36:03.226778+0000","last_undegraded":"2024-08-20T06:36:03.226778+0000","last_fullsized":"2024-08-20T06:36:03.226778+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:29:35.001592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.523940+0000","last_change":"2024-08-20T06:34:59.934386+0000","last_active":"2024-08-20T06:36:31.523940+0000","last_peered":"2024-08-20T06:36:31.523940+0000","last_clean":"2024-08-20T06:36:31.523940+0000","last_became_active":"2024-08-20T06:34:59.931079+0000","last_became_peered":"2024-08-20T06:34:59.931079+0000","last_unstale":"2024-08-20T06:36:31.523940+0000","last_undegraded":"2024-08-20T06:36:31.523940+0000","last_fullsized":"2024-08-20T06:36:31.523940+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:15:00.150462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121042+0000","last_change":"2024-08-20T06:34:57.911343+0000","last_active":"2024-08-20T06:36:03.121042+0000","last_peered":"2024-08-20T06:36:03.121042+0000","last_clean":"2024-08-20T06:36:03.121042+0000","last_became_active":"2024-08-20T06:34:57.911007+0000","last_became_peered":"2024-08-20T06:34:57.911007+0000","last_unstale":"2024-08-20T06:36:03.121042+0000","last_undegraded":"2024-08-20T06:36:03.121042+0000","last_fullsized":"2024-08-20T06:36:03.121042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:08:23.874336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534641+0000","last_change":"2024-08-20T06:34:59.929619+0000","last_active":"2024-08-20T06:36:31.534641+0000","last_peered":"2024-08-20T06:36:31.534641+0000","last_clean":"2024-08-20T06:36:31.534641+0000","last_became_active":"2024-08-20T06:34:59.929305+0000","last_became_peered":"2024-08-20T06:34:59.929305+0000","last_unstale":"2024-08-20T06:36:31.534641+0000","last_undegraded":"2024-08-20T06:36:31.534641+0000","last_fullsized":"2024-08-20T06:36:31.534641+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:10:33.726084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"73'4","reported_seq":135,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.147134+0000","last_change":"2024-08-20T06:34:57.910774+0000","last_active":"2024-08-20T06:36:56.147134+0000","last_peered":"2024-08-20T06:36:56.147134+0000","last_clean":"2024-08-20T06:36:56.147134+0000","last_became_active":"2024-08-20T06:34:57.910371+0000","last_became_peered":"2024-08-20T06:34:57.910371+0000","last_unstale":"2024-08-20T06:36:56.147134+0000","last_undegraded":"2024-08-20T06:36:56.147134+0000","last_fullsized":"2024-08-20T06:36:56.147134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:03:19.439668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"73'10","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541403+0000","last_change":"2024-08-20T06:34:59.931892+0000","last_active":"2024-08-20T06:36:31.541403+0000","last_peered":"2024-08-20T06:36:31.541403+0000","last_clean":"2024-08-20T06:36:31.541403+0000","last_became_active":"2024-08-20T06:34:59.931560+0000","last_became_peered":"2024-08-20T06:34:59.931560+0000","last_unstale":"2024-08-20T06:36:31.541403+0000","last_undegraded":"2024-08-20T06:36:31.541403+0000","last_fullsized":"2024-08-20T06:36:31.541403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:45:50.348382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122853+0000","last_change":"2024-08-20T06:34:57.920946+0000","last_active":"2024-08-20T06:36:03.122853+0000","last_peered":"2024-08-20T06:36:03.122853+0000","last_clean":"2024-08-20T06:36:03.122853+0000","last_became_active":"2024-08-20T06:34:57.920176+0000","last_became_peered":"2024-08-20T06:34:57.920176+0000","last_unstale":"2024-08-20T06:36:03.122853+0000","last_undegraded":"2024-08-20T06:36:03.122853+0000","last_fullsized":"2024-08-20T06:36:03.122853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:02:01.194084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"73'9","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531015+0000","last_change":"2024-08-20T06:36:03.227945+0000","last_active":"2024-08-20T06:36:31.531015+0000","last_peered":"2024-08-20T06:36:31.531015+0000","last_clean":"2024-08-20T06:36:31.531015+0000","last_became_active":"2024-08-20T06:36:03.227527+0000","last_became_peered":"2024-08-20T06:36:03.227527+0000","last_unstale":"2024-08-20T06:36:31.531015+0000","last_undegraded":"2024-08-20T06:36:31.531015+0000","last_fullsized":"2024-08-20T06:36:31.531015+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:54.269249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121477+0000","last_change":"2024-08-20T06:34:57.973633+0000","last_active":"2024-08-20T06:36:03.121477+0000","last_peered":"2024-08-20T06:36:03.121477+0000","last_clean":"2024-08-20T06:36:03.121477+0000","last_became_active":"2024-08-20T06:34:57.973228+0000","last_became_peered":"2024-08-20T06:34:57.973228+0000","last_unstale":"2024-08-20T06:36:03.121477+0000","last_undegraded":"2024-08-20T06:36:03.121477+0000","last_fullsized":"2024-08-20T06:36:03.121477+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:25:50.438679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"73'11","reported_seq":29,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542985+0000","last_change":"2024-08-20T06:34:59.939200+0000","last_active":"2024-08-20T06:36:31.542985+0000","last_peered":"2024-08-20T06:36:31.542985+0000","last_clean":"2024-08-20T06:36:31.542985+0000","last_became_active":"2024-08-20T06:34:59.938838+0000","last_became_peered":"2024-08-20T06:34:59.938838+0000","last_unstale":"2024-08-20T06:36:31.542985+0000","last_undegraded":"2024-08-20T06:36:31.542985+0000","last_fullsized":"2024-08-20T06:36:31.542985+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:23:29.453455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"73'35","reported_seq":301,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.428964+0000","last_change":"2024-08-20T06:36:03.346793+0000","last_active":"2024-08-20T06:36:56.428964+0000","last_peered":"2024-08-20T06:36:56.428964+0000","last_clean":"2024-08-20T06:36:56.428964+0000","last_became_active":"2024-08-20T06:36:03.340465+0000","last_became_peered":"2024-08-20T06:36:03.340465+0000","last_unstale":"2024-08-20T06:36:56.428964+0000","last_undegraded":"2024-08-20T06:36:56.428964+0000","last_fullsized":"2024-08-20T06:36:56.428964+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:07:46.208693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67197,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":222,"num_read_kb":172,"num_write":36,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":61750,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531845+0000","last_change":"2024-08-20T06:34:59.935969+0000","last_active":"2024-08-20T06:36:31.531845+0000","last_peered":"2024-08-20T06:36:31.531845+0000","last_clean":"2024-08-20T06:36:31.531845+0000","last_became_active":"2024-08-20T06:34:59.935632+0000","last_became_peered":"2024-08-20T06:34:59.935632+0000","last_unstale":"2024-08-20T06:36:31.531845+0000","last_undegraded":"2024-08-20T06:36:31.531845+0000","last_fullsized":"2024-08-20T06:36:31.531845+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:37:24.623977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"71'6","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.138299+0000","last_change":"2024-08-20T06:36:03.137915+0000","last_active":"2024-08-20T06:36:03.138299+0000","last_peered":"2024-08-20T06:36:03.138299+0000","last_clean":"2024-08-20T06:36:03.138299+0000","last_became_active":"2024-08-20T06:36:03.129079+0000","last_became_peered":"2024-08-20T06:36:03.129079+0000","last_unstale":"2024-08-20T06:36:03.138299+0000","last_undegraded":"2024-08-20T06:36:03.138299+0000","last_fullsized":"2024-08-20T06:36:03.138299+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:30:25.830576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532459+0000","last_change":"2024-08-20T06:34:59.930140+0000","last_active":"2024-08-20T06:36:31.532459+0000","last_peered":"2024-08-20T06:36:31.532459+0000","last_clean":"2024-08-20T06:36:31.532459+0000","last_became_active":"2024-08-20T06:34:59.929840+0000","last_became_peered":"2024-08-20T06:34:59.929840+0000","last_unstale":"2024-08-20T06:36:31.532459+0000","last_undegraded":"2024-08-20T06:36:31.532459+0000","last_fullsized":"2024-08-20T06:36:31.532459+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:24:50.971807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121199+0000","last_change":"2024-08-20T06:34:57.908162+0000","last_active":"2024-08-20T06:36:03.121199+0000","last_peered":"2024-08-20T06:36:03.121199+0000","last_clean":"2024-08-20T06:36:03.121199+0000","last_became_active":"2024-08-20T06:34:57.907858+0000","last_became_peered":"2024-08-20T06:34:57.907858+0000","last_unstale":"2024-08-20T06:36:03.121199+0000","last_undegraded":"2024-08-20T06:36:03.121199+0000","last_fullsized":"2024-08-20T06:36:03.121199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:29:43.848404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"73'15","reported_seq":40,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.534636+0000","last_change":"2024-08-20T06:36:03.227929+0000","last_active":"2024-08-20T06:36:31.534636+0000","last_peered":"2024-08-20T06:36:31.534636+0000","last_clean":"2024-08-20T06:36:31.534636+0000","last_became_active":"2024-08-20T06:36:03.227647+0000","last_became_peered":"2024-08-20T06:36:03.227647+0000","last_unstale":"2024-08-20T06:36:31.534636+0000","last_undegraded":"2024-08-20T06:36:31.534636+0000","last_fullsized":"2024-08-20T06:36:31.534636+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:25.422684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"73'5","reported_seq":140,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:57.159656+0000","last_change":"2024-08-20T06:34:57.914104+0000","last_active":"2024-08-20T06:36:57.159656+0000","last_peered":"2024-08-20T06:36:57.159656+0000","last_clean":"2024-08-20T06:36:57.159656+0000","last_became_active":"2024-08-20T06:34:57.913380+0000","last_became_peered":"2024-08-20T06:34:57.913380+0000","last_unstale":"2024-08-20T06:36:57.159656+0000","last_undegraded":"2024-08-20T06:36:57.159656+0000","last_fullsized":"2024-08-20T06:36:57.159656+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:23.317222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1472,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":5,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.532941+0000","last_change":"2024-08-20T06:34:59.929243+0000","last_active":"2024-08-20T06:36:31.532941+0000","last_peered":"2024-08-20T06:36:31.532941+0000","last_clean":"2024-08-20T06:36:31.532941+0000","last_became_active":"2024-08-20T06:34:59.928893+0000","last_became_peered":"2024-08-20T06:34:59.928893+0000","last_unstale":"2024-08-20T06:36:31.532941+0000","last_undegraded":"2024-08-20T06:36:31.532941+0000","last_fullsized":"2024-08-20T06:36:31.532941+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:34:14.496025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"71'15","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124876+0000","last_change":"2024-08-20T06:34:57.974765+0000","last_active":"2024-08-20T06:36:03.124876+0000","last_peered":"2024-08-20T06:36:03.124876+0000","last_clean":"2024-08-20T06:36:03.124876+0000","last_became_active":"2024-08-20T06:34:57.974370+0000","last_became_peered":"2024-08-20T06:34:57.974370+0000","last_unstale":"2024-08-20T06:36:03.124876+0000","last_undegraded":"2024-08-20T06:36:03.124876+0000","last_fullsized":"2024-08-20T06:36:03.124876+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:48:24.278527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542040+0000","last_change":"2024-08-20T06:34:59.937985+0000","last_active":"2024-08-20T06:36:31.542040+0000","last_peered":"2024-08-20T06:36:31.542040+0000","last_clean":"2024-08-20T06:36:31.542040+0000","last_became_active":"2024-08-20T06:34:59.937584+0000","last_became_peered":"2024-08-20T06:34:59.937584+0000","last_unstale":"2024-08-20T06:36:31.542040+0000","last_undegraded":"2024-08-20T06:36:31.542040+0000","last_fullsized":"2024-08-20T06:36:31.542040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:17:25.754559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125916+0000","last_change":"2024-08-20T06:34:57.915783+0000","last_active":"2024-08-20T06:36:03.125916+0000","last_peered":"2024-08-20T06:36:03.125916+0000","last_clean":"2024-08-20T06:36:03.125916+0000","last_became_active":"2024-08-20T06:34:57.915402+0000","last_became_peered":"2024-08-20T06:34:57.915402+0000","last_unstale":"2024-08-20T06:36:03.125916+0000","last_undegraded":"2024-08-20T06:36:03.125916+0000","last_fullsized":"2024-08-20T06:36:03.125916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:49:47.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527503+0000","last_change":"2024-08-20T06:34:59.937560+0000","last_active":"2024-08-20T06:36:31.527503+0000","last_peered":"2024-08-20T06:36:31.527503+0000","last_clean":"2024-08-20T06:36:31.527503+0000","last_became_active":"2024-08-20T06:34:59.937197+0000","last_became_peered":"2024-08-20T06:34:59.937197+0000","last_unstale":"2024-08-20T06:36:31.527503+0000","last_undegraded":"2024-08-20T06:36:31.527503+0000","last_fullsized":"2024-08-20T06:36:31.527503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:28:13.473132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.227139+0000","last_change":"2024-08-20T06:34:57.920553+0000","last_active":"2024-08-20T06:36:03.227139+0000","last_peered":"2024-08-20T06:36:03.227139+0000","last_clean":"2024-08-20T06:36:03.227139+0000","last_became_active":"2024-08-20T06:34:57.920219+0000","last_became_peered":"2024-08-20T06:34:57.920219+0000","last_unstale":"2024-08-20T06:36:03.227139+0000","last_undegraded":"2024-08-20T06:36:03.227139+0000","last_fullsized":"2024-08-20T06:36:03.227139+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:23:53.142212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530907+0000","last_change":"2024-08-20T06:34:59.940948+0000","last_active":"2024-08-20T06:36:31.530907+0000","last_peered":"2024-08-20T06:36:31.530907+0000","last_clean":"2024-08-20T06:36:31.530907+0000","last_became_active":"2024-08-20T06:34:59.940604+0000","last_became_peered":"2024-08-20T06:34:59.940604+0000","last_unstale":"2024-08-20T06:36:31.530907+0000","last_undegraded":"2024-08-20T06:36:31.530907+0000","last_fullsized":"2024-08-20T06:36:31.530907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:48:26.852607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"71'1","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.121585+0000","last_change":"2024-08-20T06:34:57.912635+0000","last_active":"2024-08-20T06:36:03.121585+0000","last_peered":"2024-08-20T06:36:03.121585+0000","last_clean":"2024-08-20T06:36:03.121585+0000","last_became_active":"2024-08-20T06:34:57.912226+0000","last_became_peered":"2024-08-20T06:34:57.912226+0000","last_unstale":"2024-08-20T06:36:03.121585+0000","last_undegraded":"2024-08-20T06:36:03.121585+0000","last_fullsized":"2024-08-20T06:36:03.121585+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T14:27:16.846490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"73'7","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.531455+0000","last_change":"2024-08-20T06:36:03.132019+0000","last_active":"2024-08-20T06:36:31.531455+0000","last_peered":"2024-08-20T06:36:31.531455+0000","last_clean":"2024-08-20T06:36:31.531455+0000","last_became_active":"2024-08-20T06:36:03.126050+0000","last_became_peered":"2024-08-20T06:36:03.126050+0000","last_unstale":"2024-08-20T06:36:31.531455+0000","last_undegraded":"2024-08-20T06:36:31.531455+0000","last_fullsized":"2024-08-20T06:36:31.531455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:40:08.558872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"71'3","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122496+0000","last_change":"2024-08-20T06:34:58.147444+0000","last_active":"2024-08-20T06:36:03.122496+0000","last_peered":"2024-08-20T06:36:03.122496+0000","last_clean":"2024-08-20T06:36:03.122496+0000","last_became_active":"2024-08-20T06:34:58.147048+0000","last_became_peered":"2024-08-20T06:34:58.147048+0000","last_unstale":"2024-08-20T06:36:03.122496+0000","last_undegraded":"2024-08-20T06:36:03.122496+0000","last_fullsized":"2024-08-20T06:36:03.122496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:19:33.887099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.538900+0000","last_change":"2024-08-20T06:34:59.937018+0000","last_active":"2024-08-20T06:36:31.538900+0000","last_peered":"2024-08-20T06:36:31.538900+0000","last_clean":"2024-08-20T06:36:31.538900+0000","last_became_active":"2024-08-20T06:34:59.936703+0000","last_became_peered":"2024-08-20T06:34:59.936703+0000","last_unstale":"2024-08-20T06:36:31.538900+0000","last_undegraded":"2024-08-20T06:36:31.538900+0000","last_fullsized":"2024-08-20T06:36:31.538900+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:32:34.334052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'92","reported_seq":92,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122684+0000","last_change":"2024-08-20T06:33:48.712037+0000","last_active":"2024-08-20T06:36:03.122684+0000","last_peered":"2024-08-20T06:36:03.122684+0000","last_clean":"2024-08-20T06:36:03.122684+0000","last_became_active":"2024-08-20T06:33:48.706689+0000","last_became_peered":"2024-08-20T06:33:48.706689+0000","last_unstale":"2024-08-20T06:36:03.122684+0000","last_undegraded":"2024-08-20T06:36:03.122684+0000","last_fullsized":"2024-08-20T06:36:03.122684+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:30:39.906266+0000","last_clean_scrub_stamp":"2024-08-20T06:30:39.906266+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:01:02.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"71'3","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339216+0000","last_change":"2024-08-20T06:34:57.912957+0000","last_active":"2024-08-20T06:36:03.339216+0000","last_peered":"2024-08-20T06:36:03.339216+0000","last_clean":"2024-08-20T06:36:03.339216+0000","last_became_active":"2024-08-20T06:34:57.912364+0000","last_became_peered":"2024-08-20T06:34:57.912364+0000","last_unstale":"2024-08-20T06:36:03.339216+0000","last_undegraded":"2024-08-20T06:36:03.339216+0000","last_fullsized":"2024-08-20T06:36:03.339216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:55:48.417585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.529343+0000","last_change":"2024-08-20T06:34:59.937184+0000","last_active":"2024-08-20T06:36:31.529343+0000","last_peered":"2024-08-20T06:36:31.529343+0000","last_clean":"2024-08-20T06:36:31.529343+0000","last_became_active":"2024-08-20T06:34:59.936810+0000","last_became_peered":"2024-08-20T06:34:59.936810+0000","last_unstale":"2024-08-20T06:36:31.529343+0000","last_undegraded":"2024-08-20T06:36:31.529343+0000","last_fullsized":"2024-08-20T06:36:31.529343+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:24:33.200703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"71'3","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.135010+0000","last_change":"2024-08-20T06:36:03.134639+0000","last_active":"2024-08-20T06:36:03.135010+0000","last_peered":"2024-08-20T06:36:03.135010+0000","last_clean":"2024-08-20T06:36:03.135010+0000","last_became_active":"2024-08-20T06:36:03.132044+0000","last_became_peered":"2024-08-20T06:36:03.132044+0000","last_unstale":"2024-08-20T06:36:03.135010+0000","last_undegraded":"2024-08-20T06:36:03.135010+0000","last_fullsized":"2024-08-20T06:36:03.135010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:28:06.213640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525687+0000","last_change":"2024-08-20T06:34:59.929058+0000","last_active":"2024-08-20T06:36:31.525687+0000","last_peered":"2024-08-20T06:36:31.525687+0000","last_clean":"2024-08-20T06:36:31.525687+0000","last_became_active":"2024-08-20T06:34:59.928773+0000","last_became_peered":"2024-08-20T06:34:59.928773+0000","last_unstale":"2024-08-20T06:36:31.525687+0000","last_undegraded":"2024-08-20T06:36:31.525687+0000","last_fullsized":"2024-08-20T06:36:31.525687+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:28:13.998185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"73'5","reported_seq":136,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:56.157371+0000","last_change":"2024-08-20T06:34:57.973223+0000","last_active":"2024-08-20T06:36:56.157371+0000","last_peered":"2024-08-20T06:36:56.157371+0000","last_clean":"2024-08-20T06:36:56.157371+0000","last_became_active":"2024-08-20T06:34:57.972867+0000","last_became_peered":"2024-08-20T06:34:57.972867+0000","last_unstale":"2024-08-20T06:36:56.157371+0000","last_undegraded":"2024-08-20T06:36:56.157371+0000","last_fullsized":"2024-08-20T06:36:56.157371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:04.509665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541026+0000","last_change":"2024-08-20T06:34:59.923771+0000","last_active":"2024-08-20T06:36:31.541026+0000","last_peered":"2024-08-20T06:36:31.541026+0000","last_clean":"2024-08-20T06:36:31.541026+0000","last_became_active":"2024-08-20T06:34:59.923257+0000","last_became_peered":"2024-08-20T06:34:59.923257+0000","last_unstale":"2024-08-20T06:36:31.541026+0000","last_undegraded":"2024-08-20T06:36:31.541026+0000","last_fullsized":"2024-08-20T06:36:31.541026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:39:52.334678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122243+0000","last_change":"2024-08-20T06:34:57.911676+0000","last_active":"2024-08-20T06:36:03.122243+0000","last_peered":"2024-08-20T06:36:03.122243+0000","last_clean":"2024-08-20T06:36:03.122243+0000","last_became_active":"2024-08-20T06:34:57.911330+0000","last_became_peered":"2024-08-20T06:34:57.911330+0000","last_unstale":"2024-08-20T06:36:03.122243+0000","last_undegraded":"2024-08-20T06:36:03.122243+0000","last_fullsized":"2024-08-20T06:36:03.122243+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:30:57.090372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.530300+0000","last_change":"2024-08-20T06:34:59.932238+0000","last_active":"2024-08-20T06:36:31.530300+0000","last_peered":"2024-08-20T06:36:31.530300+0000","last_clean":"2024-08-20T06:36:31.530300+0000","last_became_active":"2024-08-20T06:34:59.930981+0000","last_became_peered":"2024-08-20T06:34:59.930981+0000","last_unstale":"2024-08-20T06:36:31.530300+0000","last_undegraded":"2024-08-20T06:36:31.530300+0000","last_fullsized":"2024-08-20T06:36:31.530300+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:14.485684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123951+0000","last_change":"2024-08-20T06:34:57.974677+0000","last_active":"2024-08-20T06:36:03.123951+0000","last_peered":"2024-08-20T06:36:03.123951+0000","last_clean":"2024-08-20T06:36:03.123951+0000","last_became_active":"2024-08-20T06:34:57.974319+0000","last_became_peered":"2024-08-20T06:34:57.974319+0000","last_unstale":"2024-08-20T06:36:03.123951+0000","last_undegraded":"2024-08-20T06:36:03.123951+0000","last_fullsized":"2024-08-20T06:36:03.123951+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:17:05.268982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.537954+0000","last_change":"2024-08-20T06:34:59.929730+0000","last_active":"2024-08-20T06:36:31.537954+0000","last_peered":"2024-08-20T06:36:31.537954+0000","last_clean":"2024-08-20T06:36:31.537954+0000","last_became_active":"2024-08-20T06:34:59.929375+0000","last_became_peered":"2024-08-20T06:34:59.929375+0000","last_unstale":"2024-08-20T06:36:31.537954+0000","last_undegraded":"2024-08-20T06:36:31.537954+0000","last_fullsized":"2024-08-20T06:36:31.537954+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:37:11.338354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"71'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131744+0000","last_change":"2024-08-20T06:36:03.131209+0000","last_active":"2024-08-20T06:36:03.131744+0000","last_peered":"2024-08-20T06:36:03.131744+0000","last_clean":"2024-08-20T06:36:03.131744+0000","last_became_active":"2024-08-20T06:36:03.127637+0000","last_became_peered":"2024-08-20T06:36:03.127637+0000","last_unstale":"2024-08-20T06:36:03.131744+0000","last_undegraded":"2024-08-20T06:36:03.131744+0000","last_fullsized":"2024-08-20T06:36:03.131744+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:06:16.856600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.526170+0000","last_change":"2024-08-20T06:34:59.927057+0000","last_active":"2024-08-20T06:36:31.526170+0000","last_peered":"2024-08-20T06:36:31.526170+0000","last_clean":"2024-08-20T06:36:31.526170+0000","last_became_active":"2024-08-20T06:34:59.926363+0000","last_became_peered":"2024-08-20T06:34:59.926363+0000","last_unstale":"2024-08-20T06:36:31.526170+0000","last_undegraded":"2024-08-20T06:36:31.526170+0000","last_fullsized":"2024-08-20T06:36:31.526170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:23:31.071524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"73'11","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123684+0000","last_change":"2024-08-20T06:34:57.913323+0000","last_active":"2024-08-20T06:36:03.123684+0000","last_peered":"2024-08-20T06:36:03.123684+0000","last_clean":"2024-08-20T06:36:03.123684+0000","last_became_active":"2024-08-20T06:34:57.912908+0000","last_became_peered":"2024-08-20T06:34:57.912908+0000","last_unstale":"2024-08-20T06:36:03.123684+0000","last_undegraded":"2024-08-20T06:36:03.123684+0000","last_fullsized":"2024-08-20T06:36:03.123684+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:31:22.620623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.525153+0000","last_change":"2024-08-20T06:34:59.929499+0000","last_active":"2024-08-20T06:36:31.525153+0000","last_peered":"2024-08-20T06:36:31.525153+0000","last_clean":"2024-08-20T06:36:31.525153+0000","last_became_active":"2024-08-20T06:34:59.929038+0000","last_became_peered":"2024-08-20T06:34:59.929038+0000","last_unstale":"2024-08-20T06:36:31.525153+0000","last_undegraded":"2024-08-20T06:36:31.525153+0000","last_fullsized":"2024-08-20T06:36:31.525153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:12:08.668216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.339652+0000","last_change":"2024-08-20T06:34:58.147671+0000","last_active":"2024-08-20T06:36:03.339652+0000","last_peered":"2024-08-20T06:36:03.339652+0000","last_clean":"2024-08-20T06:36:03.339652+0000","last_became_active":"2024-08-20T06:34:58.147288+0000","last_became_peered":"2024-08-20T06:34:58.147288+0000","last_unstale":"2024-08-20T06:36:03.339652+0000","last_undegraded":"2024-08-20T06:36:03.339652+0000","last_fullsized":"2024-08-20T06:36:03.339652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:58:04.526574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"73'7","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.540510+0000","last_change":"2024-08-20T06:34:59.928964+0000","last_active":"2024-08-20T06:36:31.540510+0000","last_peered":"2024-08-20T06:36:31.540510+0000","last_clean":"2024-08-20T06:36:31.540510+0000","last_became_active":"2024-08-20T06:34:59.928663+0000","last_became_peered":"2024-08-20T06:34:59.928663+0000","last_unstale":"2024-08-20T06:36:31.540510+0000","last_undegraded":"2024-08-20T06:36:31.540510+0000","last_fullsized":"2024-08-20T06:36:31.540510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:21:11.043500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.131350+0000","last_change":"2024-08-20T06:34:57.974776+0000","last_active":"2024-08-20T06:36:03.131350+0000","last_peered":"2024-08-20T06:36:03.131350+0000","last_clean":"2024-08-20T06:36:03.131350+0000","last_became_active":"2024-08-20T06:34:57.974453+0000","last_became_peered":"2024-08-20T06:34:57.974453+0000","last_unstale":"2024-08-20T06:36:03.131350+0000","last_undegraded":"2024-08-20T06:36:03.131350+0000","last_fullsized":"2024-08-20T06:36:03.131350+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:31:39.905142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"73'2","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.493499+0000","last_change":"2024-08-20T06:34:59.933839+0000","last_active":"2024-08-20T06:36:31.493499+0000","last_peered":"2024-08-20T06:36:31.493499+0000","last_clean":"2024-08-20T06:36:31.493499+0000","last_became_active":"2024-08-20T06:34:59.933519+0000","last_became_peered":"2024-08-20T06:34:59.933519+0000","last_unstale":"2024-08-20T06:36:31.493499+0000","last_undegraded":"2024-08-20T06:36:31.493499+0000","last_fullsized":"2024-08-20T06:36:31.493499+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:05:25.780679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"71'3","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.124496+0000","last_change":"2024-08-20T06:34:57.909784+0000","last_active":"2024-08-20T06:36:03.124496+0000","last_peered":"2024-08-20T06:36:03.124496+0000","last_clean":"2024-08-20T06:36:03.124496+0000","last_became_active":"2024-08-20T06:34:57.909451+0000","last_became_peered":"2024-08-20T06:34:57.909451+0000","last_unstale":"2024-08-20T06:36:03.124496+0000","last_undegraded":"2024-08-20T06:36:03.124496+0000","last_fullsized":"2024-08-20T06:36:03.124496+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:32:55.836821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.527640+0000","last_change":"2024-08-20T06:34:59.940667+0000","last_active":"2024-08-20T06:36:31.527640+0000","last_peered":"2024-08-20T06:36:31.527640+0000","last_clean":"2024-08-20T06:36:31.527640+0000","last_became_active":"2024-08-20T06:34:59.940302+0000","last_became_peered":"2024-08-20T06:34:59.940302+0000","last_unstale":"2024-08-20T06:36:31.527640+0000","last_undegraded":"2024-08-20T06:36:31.527640+0000","last_fullsized":"2024-08-20T06:36:31.527640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:57:33.658325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"71'1","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.122065+0000","last_change":"2024-08-20T06:34:57.919226+0000","last_active":"2024-08-20T06:36:03.122065+0000","last_peered":"2024-08-20T06:36:03.122065+0000","last_clean":"2024-08-20T06:36:03.122065+0000","last_became_active":"2024-08-20T06:34:57.918717+0000","last_became_peered":"2024-08-20T06:34:57.918717+0000","last_unstale":"2024-08-20T06:36:03.122065+0000","last_undegraded":"2024-08-20T06:36:03.122065+0000","last_fullsized":"2024-08-20T06:36:03.122065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T16:57:54.031514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"73'8","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.535568+0000","last_change":"2024-08-20T06:34:59.936562+0000","last_active":"2024-08-20T06:36:31.535568+0000","last_peered":"2024-08-20T06:36:31.535568+0000","last_clean":"2024-08-20T06:36:31.535568+0000","last_became_active":"2024-08-20T06:34:59.936226+0000","last_became_peered":"2024-08-20T06:34:59.936226+0000","last_unstale":"2024-08-20T06:36:31.535568+0000","last_undegraded":"2024-08-20T06:36:31.535568+0000","last_fullsized":"2024-08-20T06:36:31.535568+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:39.279670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"71'4","reported_seq":28,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125161+0000","last_change":"2024-08-20T06:34:57.920271+0000","last_active":"2024-08-20T06:36:03.125161+0000","last_peered":"2024-08-20T06:36:03.125161+0000","last_clean":"2024-08-20T06:36:03.125161+0000","last_became_active":"2024-08-20T06:34:57.919932+0000","last_became_peered":"2024-08-20T06:34:57.919932+0000","last_unstale":"2024-08-20T06:36:03.125161+0000","last_undegraded":"2024-08-20T06:36:03.125161+0000","last_fullsized":"2024-08-20T06:36:03.125161+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T18:27:14.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.539026+0000","last_change":"2024-08-20T06:34:59.939658+0000","last_active":"2024-08-20T06:36:31.539026+0000","last_peered":"2024-08-20T06:36:31.539026+0000","last_clean":"2024-08-20T06:36:31.539026+0000","last_became_active":"2024-08-20T06:34:59.939304+0000","last_became_peered":"2024-08-20T06:34:59.939304+0000","last_unstale":"2024-08-20T06:36:31.539026+0000","last_undegraded":"2024-08-20T06:36:31.539026+0000","last_fullsized":"2024-08-20T06:36:31.539026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:08:51.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"73'15","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.542524+0000","last_change":"2024-08-20T06:34:59.938395+0000","last_active":"2024-08-20T06:36:31.542524+0000","last_peered":"2024-08-20T06:36:31.542524+0000","last_clean":"2024-08-20T06:36:31.542524+0000","last_became_active":"2024-08-20T06:34:59.938056+0000","last_became_peered":"2024-08-20T06:34:59.938056+0000","last_unstale":"2024-08-20T06:36:31.542524+0000","last_undegraded":"2024-08-20T06:36:31.542524+0000","last_fullsized":"2024-08-20T06:36:31.542524+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T17:28:14.912324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"71'2","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126001+0000","last_change":"2024-08-20T06:34:57.914005+0000","last_active":"2024-08-20T06:36:03.126001+0000","last_peered":"2024-08-20T06:36:03.126001+0000","last_clean":"2024-08-20T06:36:03.126001+0000","last_became_active":"2024-08-20T06:34:57.913671+0000","last_became_peered":"2024-08-20T06:34:57.913671+0000","last_unstale":"2024-08-20T06:36:03.126001+0000","last_undegraded":"2024-08-20T06:36:03.126001+0000","last_fullsized":"2024-08-20T06:36:03.126001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T15:10:55.332269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"73'9","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541846+0000","last_change":"2024-08-20T06:34:59.931325+0000","last_active":"2024-08-20T06:36:31.541846+0000","last_peered":"2024-08-20T06:36:31.541846+0000","last_clean":"2024-08-20T06:36:31.541846+0000","last_became_active":"2024-08-20T06:34:59.931028+0000","last_became_peered":"2024-08-20T06:34:59.931028+0000","last_unstale":"2024-08-20T06:36:31.541846+0000","last_undegraded":"2024-08-20T06:36:31.541846+0000","last_fullsized":"2024-08-20T06:36:31.541846+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:47:34.647142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"71'4","reported_seq":27,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.126507+0000","last_change":"2024-08-20T06:34:57.918377+0000","last_active":"2024-08-20T06:36:03.126507+0000","last_peered":"2024-08-20T06:36:03.126507+0000","last_clean":"2024-08-20T06:36:03.126507+0000","last_became_active":"2024-08-20T06:34:57.918046+0000","last_became_peered":"2024-08-20T06:34:57.918046+0000","last_unstale":"2024-08-20T06:36:03.126507+0000","last_undegraded":"2024-08-20T06:36:03.126507+0000","last_fullsized":"2024-08-20T06:36:03.126507+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:28:33.229515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"73'6","reported_seq":24,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.541363+0000","last_change":"2024-08-20T06:34:59.931041+0000","last_active":"2024-08-20T06:36:31.541363+0000","last_peered":"2024-08-20T06:36:31.541363+0000","last_clean":"2024-08-20T06:36:31.541363+0000","last_became_active":"2024-08-20T06:34:59.930751+0000","last_became_peered":"2024-08-20T06:34:59.930751+0000","last_unstale":"2024-08-20T06:36:31.541363+0000","last_undegraded":"2024-08-20T06:36:31.541363+0000","last_fullsized":"2024-08-20T06:36:31.541363+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:31:04.181293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"71'2","reported_seq":26,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.125324+0000","last_change":"2024-08-20T06:34:57.973445+0000","last_active":"2024-08-20T06:36:03.125324+0000","last_peered":"2024-08-20T06:36:03.125324+0000","last_clean":"2024-08-20T06:36:03.125324+0000","last_became_active":"2024-08-20T06:34:57.972896+0000","last_became_peered":"2024-08-20T06:34:57.972896+0000","last_unstale":"2024-08-20T06:36:03.125324+0000","last_undegraded":"2024-08-20T06:36:03.125324+0000","last_fullsized":"2024-08-20T06:36:03.125324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:25:01.109248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"73'5","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:31.528633+0000","last_change":"2024-08-20T06:34:59.935769+0000","last_active":"2024-08-20T06:36:31.528633+0000","last_peered":"2024-08-20T06:36:31.528633+0000","last_clean":"2024-08-20T06:36:31.528633+0000","last_became_active":"2024-08-20T06:34:59.935477+0000","last_became_peered":"2024-08-20T06:34:59.935477+0000","last_unstale":"2024-08-20T06:36:31.528633+0000","last_undegraded":"2024-08-20T06:36:31.528633+0000","last_fullsized":"2024-08-20T06:36:31.528633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:58.901370+0000","last_clean_scrub_stamp":"2024-08-20T06:34:58.901370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:40:16.714694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"71'9","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-08-20T06:36:03.123864+0000","last_change":"2024-08-20T06:34:57.913499+0000","last_active":"2024-08-20T06:36:03.123864+0000","last_peered":"2024-08-20T06:36:03.123864+0000","last_clean":"2024-08-20T06:36:03.123864+0000","last_became_active":"2024-08-20T06:34:57.912840+0000","last_became_peered":"2024-08-20T06:34:57.912840+0000","last_unstale":"2024-08-20T06:36:03.123864+0000","last_undegraded":"2024-08-20T06:36:03.123864+0000","last_fullsized":"2024-08-20T06:36:03.123864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T06:34:56.891846+0000","last_clean_scrub_stamp":"2024-08-20T06:34:56.891846+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T13:53:36.560888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72184,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":672,"num_read_kb":408,"num_write":174,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":61806,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":65,"seq":279172874277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":385735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":60,"seq":257698037801,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28492,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716660,"statfs":{"total":95995035648,"available":95965859840,"internally_reserved":0,"allocated":1503232,"data_stored":1045241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":56,"seq":240518168622,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":819200,"data_stored":386032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":51,"seq":219043332148,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":811008,"data_stored":385813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":45,"seq":193273528376,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32560,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712592,"statfs":{"total":95995035648,"available":95961694208,"internally_reserved":0,"allocated":1409024,"data_stored":978360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":39,"seq":167503724604,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717344,"statfs":{"total":95995035648,"available":95966560256,"internally_reserved":0,"allocated":802816,"data_stored":385304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":34,"seq":146028888129,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1421312,"data_stored":977551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":29,"seq":124554051655,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713104,"statfs":{"total":95995035648,"available":95962218496,"internally_reserved":0,"allocated":884736,"data_stored":453822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":23,"seq":98784247882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":385498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444112,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36140,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709012,"statfs":{"total":95995035648,"available":95958028288,"internally_reserved":0,"allocated":815104,"data_stored":386413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":12,"seq":51539607634,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":447589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":386575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T06:37:00.655 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:37:00.898 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:01.280 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:01 smithi107 ceph-mon[18495]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-20T06:37:01.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:01 smithi107 ceph-mon[18495]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:01.475 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:01 smithi107 ceph-mon[18495]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:01.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 -- 172.21.15.107:0/3359393747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 msgr2=0x7f481810f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 --2- 172.21.15.107:0/3359393747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481810f660 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4810009ca0 tx=0x7f481002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 -- 172.21.15.107:0/3359393747 shutdown_connections 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 --2- 172.21.15.107:0/3359393747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481810f660 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 --2- 172.21.15.107:0/3359393747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 0x7f4818108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 --2- 172.21.15.107:0/3359393747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 0x7f4818102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 -- 172.21.15.107:0/3359393747 >> 172.21.15.107:0/3359393747 conn(0x7f48180fe540 msgr2=0x7f4818100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 -- 172.21.15.107:0/3359393747 shutdown_connections 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 -- 172.21.15.107:0/3359393747 wait complete. 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.359+0000 7f481fe24640 1 Processor -- start 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 -- start start 2024-08-20T06:37:01.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 0x7f4818198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 0x7f4818199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f48181129e0 con 0x7f4818108e60 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4818112860 con 0x7f4818102ad0 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481fe24640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4818112b60 con 0x7f48181084e0 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60260/0 (socket says 172.21.15.107:60260) 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 -- 172.21.15.107:0/1793772178 learned_addr learned my addr 172.21.15.107:0/1793772178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481dbc0640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 0x7f4818198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481d3bf640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 0x7f4818199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:01.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 -- 172.21.15.107:0/1793772178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 msgr2=0x7f4818199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 0x7f4818199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 -- 172.21.15.107:0/1793772178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 msgr2=0x7f4818198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 0x7f4818198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 -- 172.21.15.107:0/1793772178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481819f080 con 0x7f4818108e60 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f481e3c1640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4810005cc0 tx=0x7f4810032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.360+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810032e00 con 0x7f4818108e60 2024-08-20T06:37:01.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.361+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f48100044d0 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.361+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481819f310 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.361+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810030b70 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.361+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181a5a30 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.362+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4818103d80 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.363+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f4810004680 con 0x7f4818108e60 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.363+0000 7f480effd640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 0x7f48040b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.363+0000 7f481dbc0640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 0x7f48040b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:01.480 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.364+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4810105220 con 0x7f4818108e60 2024-08-20T06:37:01.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.364+0000 7f481dbc0640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 0x7f48040b61a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f481400bbe0 tx=0x7f481400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:01.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.367+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f48100cc820 con 0x7f4818108e60 2024-08-20T06:37:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:01 smithi193 ceph-mon[21944]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-20T06:37:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:01 smithi193 ceph-mon[21944]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:01.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:01 smithi193 ceph-mon[21944]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:01.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.579+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f481810ac40 con 0x7f4818108e60 2024-08-20T06:37:01.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.579+0000 7f480effd640 1 -- 172.21.15.107:0/1793772178 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7f48100cc120 con 0x7f4818108e60 2024-08-20T06:37:01.581 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:37:01.581 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"btime":"2024-08-20T06:36:56:241712+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:36:56.009554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:37:01.582 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-08-20T06:37:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 msgr2=0x7f48040b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 0x7f48040b61a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f481400bbe0 tx=0x7f481400b390 comp rx=0 tx=0).stop 2024-08-20T06:37:01.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 msgr2=0x7f481819e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4810005cc0 tx=0x7f4810032a10 comp rx=0 tx=0).stop 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 shutdown_connections 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4818108e60 0x7f481819e8b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f48040b3ce0 0x7f48040b61a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f48181084e0 0x7f4818199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.584 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 --2- 172.21.15.107:0/1793772178 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4818102ad0 0x7f4818198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 >> 172.21.15.107:0/1793772178 conn(0x7f48180fe540 msgr2=0x7f48180feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 shutdown_connections 2024-08-20T06:37:01.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:01.582+0000 7f481fe24640 1 -- 172.21.15.107:0/1793772178 wait complete. 2024-08-20T06:37:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:01 smithi183 ceph-mon[22204]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1023 B/s wr, 6 op/s 2024-08-20T06:37:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:01 smithi183 ceph-mon[22204]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:01.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:01 smithi183 ceph-mon[22204]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:02.302 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:02 smithi107 ceph-mon[18495]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:02.303 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:02 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1793772178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:02.330 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:37:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:02 smithi193 ceph-mon[21944]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:02 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1793772178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:02.575 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:02 smithi183 ceph-mon[22204]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T06:37:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:02 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1793772178' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:03.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 -- 172.21.15.107:0/613329581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2810ce40 msgr2=0x7f5c2810f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:03.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/613329581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2810ce40 0x7f5c2810f2a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5c20009ca0 tx=0x7f5c2002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 -- 172.21.15.107:0/613329581 shutdown_connections 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/613329581 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c2810ce40 0x7f5c2810f2a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/613329581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 0x7f5c2810c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/613329581 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c281039e0 0x7f5c28103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 -- 172.21.15.107:0/613329581 >> 172.21.15.107:0/613329581 conn(0x7f5c280fd440 msgr2=0x7f5c280ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 -- 172.21.15.107:0/613329581 shutdown_connections 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.037+0000 7f5c2fe81640 1 -- 172.21.15.107:0/613329581 wait complete. 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 Processor -- start 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 -- start start 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 0x7f5c281993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c2810ce40 0x7f5c2819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:03.048 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5c281129f0 con 0x7f5c281039e0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28112870 con 0x7f5c2810ce40 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2fe81640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5c28112b70 con 0x7f5c28104300 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2dc1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2dc1d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60266/0 (socket says 172.21.15.107:60266) 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2dc1d640 1 -- 172.21.15.107:0/3645628663 learned_addr learned my addr 172.21.15.107:0/3645628663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.038+0000 7f5c2e41e640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c2810ce40 0x7f5c2819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2d41c640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 0x7f5c281993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 -- 172.21.15.107:0/3645628663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 msgr2=0x7f5c281993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 0x7f5c281993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 -- 172.21.15.107:0/3645628663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c2810ce40 msgr2=0x7f5c2819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c2810ce40 0x7f5c2819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 -- 172.21.15.107:0/3645628663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2819ef30 con 0x7f5c281039e0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2dc1d640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400b4a0 tx=0x7f5c2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:03.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24014070 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2819f220 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5c24010aa0 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24017b70 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.039+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c281a58e0 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.040+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c28103dc0 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.041+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f5c24017070 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.042+0000 7f5c1effd640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 0x7f5c140b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.042+0000 7f5c2d41c640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 0x7f5c140b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.042+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5c240e34c0 con 0x7f5c281039e0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.042+0000 7f5c2d41c640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 0x7f5c140b6120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005fd0 tx=0x7f5c18005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:03.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.045+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5c240aaa80 con 0x7f5c281039e0 2024-08-20T06:37:03.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.220+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c281a5d00 con 0x7f5c281039e0 2024-08-20T06:37:03.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.222+0000 7f5c1effd640 1 -- 172.21.15.107:0/3645628663 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7f5c240aa380 con 0x7f5c281039e0 2024-08-20T06:37:03.223 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:37:03.223 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 msgr2=0x7f5c140b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 0x7f5c140b6120 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5c18005fd0 tx=0x7f5c18005d80 comp rx=0 tx=0).stop 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 msgr2=0x7f5c28198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c2400b4a0 tx=0x7f5c2400b960 comp rx=0 tx=0).stop 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 shutdown_connections 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5c2810ce40 0x7f5c2819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5c140b3c60 0x7f5c140b6120 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c28104300 0x7f5c281993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 --2- 172.21.15.107:0/3645628663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c281039e0 0x7f5c28198e80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:03.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 >> 172.21.15.107:0/3645628663 conn(0x7f5c280fd440 msgr2=0x7f5c2810f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:03.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 shutdown_connections 2024-08-20T06:37:03.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:03.224+0000 7f5c2fe81640 1 -- 172.21.15.107:0/3645628663 wait complete. 2024-08-20T06:37:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:03 smithi193 ceph-mon[21944]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:03 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3645628663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:37:03.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:03 smithi107 ceph-mon[18495]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:03.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:03 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3645628663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:37:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:03 smithi183 ceph-mon[22204]: pgmap v333: 97 pgs: 97 active+clean; 648 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:03.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:03 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3645628663' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:37:04.022 DEBUG:tasks.fwd_scrub:Yielding 2024-08-20T06:37:04.022 INFO:teuthology.run_tasks:Running task workunit... 2024-08-20T06:37:04.036 INFO:tasks.workunit:Pulling workunits from ref ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:37:04.037 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-20T06:37:04.037 INFO:tasks.workunit:timeout=3h 2024-08-20T06:37:04.037 INFO:tasks.workunit:cleanup=True 2024-08-20T06:37:04.037 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-20T06:37:04.038 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-08-20T06:37:04.038 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:37:04.038 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 79h/121d Inode: 1 Links: 2 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:fusefs_t:s0 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-08-20 06:35:15.167484310 +0000 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-08-20 06:35:15.167484310 +0000 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-08-20 06:36:55.647595693 +0000 2024-08-20T06:37:04.054 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: - 2024-08-20T06:37:04.054 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-20T06:37:04.055 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-20T06:37:04.126 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T06:37:04.181 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-20T06:37:04.293 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:04.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/4137781711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d4410f110 msgr2=0x7f5d44111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.766 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/4137781711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d4410f110 0x7f5d44111500 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009ca0 tx=0x7f5d3c02a950 comp rx=0 tx=0).stop 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/4137781711 shutdown_connections 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/4137781711 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d4410f110 0x7f5d44111500 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/4137781711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d44101e80 0x7f5d4410ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/4137781711 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d44101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/4137781711 >> 172.21.15.107:0/4137781711 conn(0x7f5d440fd410 msgr2=0x7f5d440ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/4137781711 shutdown_connections 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.763+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/4137781711 wait complete. 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 Processor -- start 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 -- start start 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d4419b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.767 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4410f110 0x7f5d441a0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44114b40 con 0x7f5d44101e80 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441149c0 con 0x7f5d44101560 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4b4f1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d44114cc0 con 0x7f5d4410f110 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d48a8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d48a8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60282/0 (socket says 172.21.15.107:60282) 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1387392327 learned_addr learned my addr 172.21.15.107:0/1387392327 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4410f110 0x7f5d441a0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.764+0000 7f5d4928d640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d4419b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1387392327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4410f110 msgr2=0x7f5d441a0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4410f110 0x7f5d441a0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1387392327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 msgr2=0x7f5d4419b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d4419b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1387392327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d441a1400 con 0x7f5d44101e80 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d4928d640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d4419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c970 tx=0x7f5d3400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d3a7fc640 1 -- 172.21.15.107:0/1387392327 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34007bb0 con 0x7f5d44101e80 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d441a16f0 con 0x7f5d44101e80 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d3a7fc640 1 -- 172.21.15.107:0/1387392327 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d34007d50 con 0x7f5d44101e80 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d3a7fc640 1 -- 172.21.15.107:0/1387392327 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34004e40 con 0x7f5d44101e80 2024-08-20T06:37:04.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.765+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d441a7db0 con 0x7f5d44101e80 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.767+0000 7f5d3a7fc640 1 -- 172.21.15.107:0/1387392327 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f5d34004fe0 con 0x7f5d44101e80 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 Processor -- start 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 -- start start 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d3a7fc640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 0x7f5d300b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb33b0 0x5590befb3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befbf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5590bef29940 con 0x5590befbfc60 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5590bef18e40 con 0x5590befb3cd0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5590bef1ce80 con 0x5590befb33b0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.768+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 0x7f5d300b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60290/0 (socket says 172.21.15.107:60290) 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 -- 172.21.15.107:0/2698200033 learned_addr learned my addr 172.21.15.107:0/2698200033 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befbf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d4928d640 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb33b0 0x5590befb3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d3a7fc640 1 -- 172.21.15.107:0/1387392327 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d340e3750 con 0x7f5d44101e80 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 -- 172.21.15.107:0/2698200033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb33b0 msgr2=0x5590befb3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb33b0 0x5590befb3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.775 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 -- 172.21.15.107:0/2698200033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 msgr2=0x5590befbf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befbf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 -- 172.21.15.107:0/2698200033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590beeb5520 con 0x5590befbfc60 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 0x7f5d300b6120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c035000 tx=0x7f5d3c031da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005fb0 tx=0x7f5d3c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c5f9092fabeb56 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d2f7fe640 1 -- 172.21.15.107:0/2698200033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c034040 con 0x5590befbfc60 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d2f7fe640 1 -- 172.21.15.107:0/2698200033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d3c003db0 con 0x5590befbfc60 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.769+0000 7f5d2f7fe640 1 -- 172.21.15.107:0/2698200033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c004450 con 0x5590befbfc60 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/2698200033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 msgr2=0x5590befc2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c005fb0 tx=0x7f5d3c048000 comp rx=0 tx=0).stop 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/2698200033 shutdown_connections 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befbfc60 0x5590befc2050 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befbf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.776 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/2698200033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb33b0 0x5590befb3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/2698200033 >> 172.21.15.107:0/2698200033 conn(0x5590befb11e0 msgr2=0x5590befb2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/2698200033 shutdown_connections 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/2698200033 wait complete. 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 -- start start 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.770+0000 7f5d4d8ea740 1 Processor -- start 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 0x5590befbd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 0x5590befbd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5590bed7b890 con 0x5590befb33b0 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5590bee9f700 con 0x5590befbfc60 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5590befc2950 con 0x5590befb3cd0 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4d8ea740 10 client.0 _ll_register_callbacks cb 0x5590bef146f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:04.777 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60296/0 (socket says 172.21.15.107:60296) 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 -- 172.21.15.107:0/3657464832 learned_addr learned my addr 172.21.15.107:0/3657464832 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 0x5590befbd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 0x5590befbd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 -- 172.21.15.107:0/3657464832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 msgr2=0x5590befbd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 0x5590befbd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 -- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 msgr2=0x5590befbd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 0x5590befbd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 -- 172.21.15.107:0/3657464832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590bec78930 con 0x5590befb33b0 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 0x5590befbd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 0x5590befbd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:04.778 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d4928d640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5d40011a30 tx=0x7f5d40011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4b2b2d42d69ba8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.771+0000 7f5d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40004460 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d4000f9a0 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3c003db0 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d4000fba0 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590befce5a0 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d3c003db0 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d40018070 con 0x5590befb33b0 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 20 client.15417 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:04.779 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590befcfb60 0x5590befd1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 4 client.15417 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.772+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590bef2b890 con 0x5590befcfb60 2024-08-20T06:37:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.773+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590befcfb60 0x5590befd1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.773+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590befcfb60 0x5590befd1fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfd99e4d11001cb server_cookie=2693ad2238491cb2 in_seq=0 out_seq=0 2024-08-20T06:37:04.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.773+0000 7f5d2cff9640 10 client.15417 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d2cff9640 1 -- 172.21.15.107:0/3657464832 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590bef2b890 con 0x5590befcfb60 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d2cff9640 10 client.15417 handle_command_reply: tid=0 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 client.15417 shutdown 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 msgr2=0x5590befbcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5d40011a30 tx=0x7f5d40011ef0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 shutdown_connections 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befbfc60 0x5590befbd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590befcfb60 0x5590befd1fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befb3cd0 0x5590befbd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/3657464832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befbcc50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 >> 172.21.15.107:0/3657464832 conn(0x5590befb11e0 msgr2=0x5590befb6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d2cff9640 0 client.15417 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 shutdown_connections 2024-08-20T06:37:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/3657464832 wait complete. 2024-08-20T06:37:04.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.774+0000 7f5d4d8ea740 20 client.15417 trim_cache size 0 max 16384 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 Processor -- start 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 -- start start 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befd78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befd7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b2af0 con 0x5590befb33b0 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441b03c0 con 0x5590befb3cd0 2024-08-20T06:37:04.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441ae1f0 con 0x5590befbfc60 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4928d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4928d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60312/0 (socket says 172.21.15.107:60312) 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.791+0000 7f5d4928d640 1 -- 172.21.15.107:0/4193021560 learned_addr learned my addr 172.21.15.107:0/4193021560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befd7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 -- 172.21.15.107:0/4193021560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 msgr2=0x5590befd7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befd7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befd78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 -- 172.21.15.107:0/4193021560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 msgr2=0x5590befd78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befd78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 -- 172.21.15.107:0/4193021560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590befc2950 con 0x5590befb33b0 2024-08-20T06:37:04.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d4928d640 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d40004af0 tx=0x7f5d40008840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e507f903c8be8293 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d2effd640 1 -- 172.21.15.107:0/4193021560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000fd40 con 0x5590befb33b0 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d2effd640 1 -- 172.21.15.107:0/4193021560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d400046f0 con 0x5590befb33b0 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.792+0000 7f5d2effd640 1 -- 172.21.15.107:0/4193021560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d40003670 con 0x5590befb33b0 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/4193021560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 msgr2=0x5590befb7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5d40004af0 tx=0x7f5d40008840 comp rx=0 tx=0).stop 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/4193021560 shutdown_connections 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befd7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb3cd0 0x5590befd78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/4193021560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb33b0 0x5590befb7470 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/4193021560 >> 172.21.15.107:0/4193021560 conn(0x5590befb11e0 msgr2=0x5590befcb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/4193021560 shutdown_connections 2024-08-20T06:37:04.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/4193021560 wait complete. 2024-08-20T06:37:04.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 -- start start 2024-08-20T06:37:04.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.793+0000 7f5d4d8ea740 1 Processor -- start 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb33b0 0x5590befc3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befc4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441ae880 con 0x5590befb3cd0 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441afcf0 con 0x5590befb33b0 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d441aef50 con 0x5590befbfc60 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 10 client.0 _ll_register_callbacks cb 0x7f5d40011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:60322/0 (socket says 172.21.15.107:60322) 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1652871511 learned_addr learned my addr 172.21.15.107:0/1652871511 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4928d640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb33b0 0x5590befc3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d49a8e640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befc4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1652871511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 msgr2=0x5590befc4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befc4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb33b0 msgr2=0x5590befc3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb33b0 0x5590befc3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 -- 172.21.15.107:0/1652871511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590befc2440 con 0x5590befb3cd0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d48a8c640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d340aa240 tx=0x7f5d3400a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ac994f025a20db server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3401d040 con 0x5590befb3cd0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d340e1350 con 0x5590befb3cd0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.794+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d400046f0 con 0x5590befb3cd0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d340aa540 con 0x5590befb3cd0 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 10 client.15423 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:04.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590befb20d0 con 0x5590befb3cd0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d400046f0 con 0x5590befb3cd0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5d34020060 con 0x5590befb3cd0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 10 client.15423 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 20 client.15423 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590bf03b0f0 0x5590bf03d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 4 client.15423 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5590bef27330 con 0x5590bf03b0f0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.795+0000 7f5d4928d640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590bf03b0f0 0x5590bf03d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.796+0000 7f5d2cff9640 10 client.15423 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.796+0000 7f5d4928d640 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590bf03b0f0 0x5590bf03d510 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ca83b579efff0f5 server_cookie=ec02608ff01d5dd9 in_seq=0 out_seq=0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d2cff9640 1 -- 172.21.15.107:0/1652871511 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5590bef27330 con 0x5590bf03b0f0 2024-08-20T06:37:04.802 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d2cff9640 10 client.15423 handle_command_reply: tid=0 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 client.15423 shutdown 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 msgr2=0x5590befc4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5d340aa240 tx=0x7f5d3400a4b0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 shutdown_connections 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5590bf03b0f0 0x5590bf03d510 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590befbfc60 0x5590befc4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5590befb3cd0 0x5590befc4030 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 --2- 172.21.15.107:0/1652871511 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5590befb33b0 0x5590befc3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 >> 172.21.15.107:0/1652871511 conn(0x5590befb11e0 msgr2=0x5590befbb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d2cff9640 0 client.15423 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 shutdown_connections 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 1 -- 172.21.15.107:0/1652871511 wait complete. 2024-08-20T06:37:04.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.797+0000 7f5d4d8ea740 20 client.15423 trim_cache size 0 max 16384 2024-08-20T06:37:04.804 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:37:04.804 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:37:04.804 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "813f0697-6723-4f45-913a-b85bca0595cd", 2024-08-20T06:37:04.804 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:37:04.804 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 msgr2=0x7f5d300b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 0x7f5d300b6120 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c035000 tx=0x7f5d3c031da0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 msgr2=0x7f5d4419b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400c970 tx=0x7f5d3400ce30 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 shutdown_connections 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d300b3c60 0x7f5d300b6120 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d4410f110 0x7f5d441a0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d44101e80 0x7f5d4419b890 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 --2- 172.21.15.107:0/1387392327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d44101560 0x7f5d4419b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:04.805 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 >> 172.21.15.107:0/1387392327 conn(0x7f5d440fd410 msgr2=0x7f5d44110440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:04.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 shutdown_connections 2024-08-20T06:37:04.806 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:04.798+0000 7f5d4b4f1640 1 -- 172.21.15.107:0/1387392327 wait complete. 2024-08-20T06:37:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:05 smithi193 ceph-mon[21944]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-08-20T06:37:05.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:05 smithi107 ceph-mon[18495]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-08-20T06:37:05.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:37:05.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:05 smithi183 ceph-mon[22204]: pgmap v334: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-08-20T06:37:05.989 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:06.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:06 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:37:06.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:06 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:06.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:06 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:37:06.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:06 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:06.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 -- 172.21.15.107:0/3850875610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 msgr2=0x7fe94c102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 --2- 172.21.15.107:0/3850875610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c102eb0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009900 tx=0x7fe93c02a410 comp rx=0 tx=0).stop 2024-08-20T06:37:06.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 -- 172.21.15.107:0/3850875610 shutdown_connections 2024-08-20T06:37:06.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 --2- 172.21.15.107:0/3850875610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c108e60 0x7fe94c10f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 --2- 172.21.15.107:0/3850875610 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c1084e0 0x7fe94c108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 --2- 172.21.15.107:0/3850875610 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c102eb0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 -- 172.21.15.107:0/3850875610 >> 172.21.15.107:0/3850875610 conn(0x7fe94c0fe540 msgr2=0x7fe94c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 -- 172.21.15.107:0/3850875610 shutdown_connections 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.487+0000 7fe95142d640 1 -- 172.21.15.107:0/3850875610 wait complete. 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 Processor -- start 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 -- start start 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c1084e0 0x7fe94c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 0x7fe94c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1129d0 con 0x7fe94c102ad0 2024-08-20T06:37:06.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112850 con 0x7fe94c108e60 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe95142d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c112b50 con 0x7fe94c1084e0 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56446/0 (socket says 172.21.15.107:56446) 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 -- 172.21.15.107:0/3731913981 learned_addr learned my addr 172.21.15.107:0/3731913981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe9427fc640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c1084e0 0x7fe94c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 0x7fe94c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 -- 172.21.15.107:0/3731913981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c1084e0 msgr2=0x7fe94c199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c1084e0 0x7fe94c199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 -- 172.21.15.107:0/3731913981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 msgr2=0x7fe94c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 0x7fe94c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94affd640 1 -- 172.21.15.107:0/3731913981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c19ef70 con 0x7fe94c102ad0 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.488+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 0x7fe94c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:06.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe94affd640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009a20 tx=0x7fe93c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe948ff9640 1 -- 172.21.15.107:0/3731913981 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c03f030 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c19f200 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe948ff9640 1 -- 172.21.15.107:0/3731913981 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe93c0035f0 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe948ff9640 1 -- 172.21.15.107:0/3731913981 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c035070 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.489+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1a5920 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.491+0000 7fe953826740 1 Processor -- start 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.491+0000 7fe948ff9640 1 -- 172.21.15.107:0/3731913981 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7fe93c003aa0 con 0x7fe94c102ad0 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.491+0000 7fe953826740 1 -- start start 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdaaaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdaad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ffda158e0 con 0x558ffda9e830 2024-08-20T06:37:06.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ffda04de0 con 0x558ffda9f150 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe953826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ffda08e20 con 0x558ffdaab0e0 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56454/0 (socket says 172.21.15.107:56454) 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3647154887 learned_addr learned my addr 172.21.15.107:0/3647154887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94affd640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdaaaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe9427fc640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdaad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3647154887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 msgr2=0x558ffdaad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdaad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3647154887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 msgr2=0x558ffdaaaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdaaaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3647154887 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ffd764650 con 0x558ffda9e830 2024-08-20T06:37:06.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94affd640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdaaaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe9427fc640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdaad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe948ff9640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 0x7fe9380b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe944008e90 tx=0x7fe94400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0da5b381be787e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe9417fa640 1 -- 172.21.15.107:0/3647154887 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400f830 con 0x558ffda9e830 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe94affd640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 0x7fe9380b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe9417fa640 1 -- 172.21.15.107:0/3647154887 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe944018480 con 0x558ffda9e830 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe9417fa640 1 -- 172.21.15.107:0/3647154887 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400fe60 con 0x558ffda9e830 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.492+0000 7fe948ff9640 1 -- 172.21.15.107:0/3731913981 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe93c104740 con 0x7fe94c102ad0 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 -- 172.21.15.107:0/3647154887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 msgr2=0x558ffda9ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe94affd640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 0x7fe9380b6120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x558ffdaa19b0 tx=0x7fe93c02a410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe944008e90 tx=0x7fe94400d710 comp rx=0 tx=0).stop 2024-08-20T06:37:06.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 -- 172.21.15.107:0/3647154887 shutdown_connections 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdaad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdaaaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 --2- 172.21.15.107:0/3647154887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffda9ec10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 -- 172.21.15.107:0/3647154887 >> 172.21.15.107:0/3647154887 conn(0x558ffda9c660 msgr2=0x558ffda9d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 -- 172.21.15.107:0/3647154887 shutdown_connections 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.493+0000 7fe953826740 1 -- 172.21.15.107:0/3647154887 wait complete. 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 -- start start 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 Processor -- start 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdaa8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdaa8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x558ffd867700 con 0x558ffda9e830 2024-08-20T06:37:06.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x558ffd98b7c0 con 0x558ffdaab0e0 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x558ffdaadd90 con 0x558ffda9f150 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe953826740 10 client.0 _ll_register_callbacks cb 0x558ffd9fe6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56460/0 (socket says 172.21.15.107:56460) 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 -- 172.21.15.107:0/901000683 learned_addr learned my addr 172.21.15.107:0/901000683 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe9427fc640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdaa8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 -- 172.21.15.107:0/901000683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 msgr2=0x558ffdaa8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdaa8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 -- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 msgr2=0x558ffdaa8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdaa8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe94b7fe640 1 -- 172.21.15.107:0/901000683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ffdab71c0 con 0x558ffda9e830 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.494+0000 7fe9427fc640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdaa8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:06.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe944008ba0 tx=0x7fe944018ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98341ed3aad9ccf3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944018950 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558ffd9b4100 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe94400f630 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe94400f830 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 10 client.15435 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ffdab9ac0 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x558ffd9b4100 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe944005490 con 0x558ffda9e830 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 10 client.15435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 20 client.15435 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:06.529 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdabd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 4 client.15435 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.495+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ffda17830 con 0x558ffdabaf20 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.496+0000 7fe9427fc640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdabd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.496+0000 7fe9327fc640 10 client.15435 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.496+0000 7fe9427fc640 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdabd310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=343882760318d0f9 server_cookie=f41b433d034d7329 in_seq=0 out_seq=0 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe9327fc640 1 -- 172.21.15.107:0/901000683 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558ffda17830 con 0x558ffdabaf20 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe9327fc640 10 client.15435 handle_command_reply: tid=0 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 client.15435 shutdown 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 msgr2=0x558ffdaa80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe944008ba0 tx=0x7fe944018ca0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 shutdown_connections 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdaa8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdabd310 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdaa8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe9327fc640 0 client.15435 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 --2- 172.21.15.107:0/901000683 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdaa80e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 >> 172.21.15.107:0/901000683 conn(0x558ffda9c660 msgr2=0x558ffdaa2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 shutdown_connections 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 1 -- 172.21.15.107:0/901000683 wait complete. 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.497+0000 7fe953826740 20 client.15435 trim_cache size 0 max 16384 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 Processor -- start 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 -- start start 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdac2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdab3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b0930 con 0x558ffda9e830 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ae200 con 0x558ffda9f150 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe953826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ac030 con 0x558ffdaab0e0 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56466/0 (socket says 172.21.15.107:56466) 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.515+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3956798893 learned_addr learned my addr 172.21.15.107:0/3956798893 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3956798893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 msgr2=0x558ffdab3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdab3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3956798893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 msgr2=0x558ffdac2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdac2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 -- 172.21.15.107:0/3956798893 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ffda9c490 con 0x558ffda9e830 2024-08-20T06:37:06.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944008580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375c28b1a977958a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe940ff9640 1 -- 172.21.15.107:0/3956798893 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944033070 con 0x558ffda9e830 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe940ff9640 1 -- 172.21.15.107:0/3956798893 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe944020070 con 0x558ffda9e830 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.516+0000 7fe940ff9640 1 -- 172.21.15.107:0/3956798893 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94400fca0 con 0x558ffda9e830 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- 172.21.15.107:0/3956798893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 msgr2=0x558ffdac2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe944000f80 tx=0x7fe944008580 comp rx=0 tx=0).stop 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- 172.21.15.107:0/3956798893 shutdown_connections 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffdaab0e0 0x558ffdab3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffda9f150 0x558ffdac2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 --2- 172.21.15.107:0/3956798893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdac2940 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- 172.21.15.107:0/3956798893 >> 172.21.15.107:0/3956798893 conn(0x558ffda9c660 msgr2=0x558ffdaa9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- 172.21.15.107:0/3956798893 shutdown_connections 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- 172.21.15.107:0/3956798893 wait complete. 2024-08-20T06:37:06.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 -- start start 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.517+0000 7fe953826740 1 Processor -- start 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdab09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdab0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ac6c0 con 0x558ffda9e830 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1adb30 con 0x558ffdaab0e0 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1acd90 con 0x558ffda9f150 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 10 client.0 _ll_register_callbacks cb 0x7fe944019f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56470/0 (socket says 172.21.15.107:56470) 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 -- 172.21.15.107:0/2398693172 learned_addr learned my addr 172.21.15.107:0/2398693172 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:06.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdab0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 -- 172.21.15.107:0/2398693172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 msgr2=0x558ffdab09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdab09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 -- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 msgr2=0x558ffdab0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdab0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 -- 172.21.15.107:0/2398693172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ffda9d0e0 con 0x558ffda9e830 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe94b7fe640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe944005b40 tx=0x7fe9440245b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a92a722b9ec9c9f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe9327fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944033040 con 0x558ffda9e830 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe944020070 con 0x558ffda9e830 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe94400f9d0 con 0x558ffda9e830 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.518+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe94400fbd0 con 0x558ffda9e830 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 10 client.15441 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:06.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558ffdb25590 con 0x558ffda9e830 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe944020070 con 0x558ffda9e830 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe944024730 con 0x558ffda9e830 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 10 client.15441 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 10 client.15441 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 20 client.15441 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdb27cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 4 client.15441 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558ffdaa9050 con 0x558ffdabaf20 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.519+0000 7fe9427fc640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdb27cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe9327fc640 10 client.15441 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe9427fc640 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdb27cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a8e9f3c28b659e2 server_cookie=beb47d627c5876c7 in_seq=0 out_seq=0 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe9327fc640 1 -- 172.21.15.107:0/2398693172 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558ffdaa9050 con 0x558ffdabaf20 2024-08-20T06:37:06.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe9327fc640 10 client.15441 handle_command_reply: tid=0 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 client.15441 shutdown 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 msgr2=0x558ffdab3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe944005b40 tx=0x7fe9440245b0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 shutdown_connections 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x558ffdaab0e0 0x558ffdab0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x558ffdabaf20 0x558ffdb27cc0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558ffda9f150 0x558ffdab09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe9327fc640 0 client.15441 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 --2- 172.21.15.107:0/2398693172 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x558ffda9e830 0x558ffdab3480 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 >> 172.21.15.107:0/2398693172 conn(0x558ffda9c660 msgr2=0x558ffdaa19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 shutdown_connections 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 1 -- 172.21.15.107:0/2398693172 wait complete. 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.520+0000 7fe953826740 20 client.15441 trim_cache size 0 max 16384 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout: "813f0697-6723-4f45-913a-b85bca0595cd": { 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:37:06.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "813f0697-6723-4f45-913a-b85bca0595cd", 2024-08-20T06:37:06.539 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:37:06.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:37:06.539 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:37:06.539 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:37:06.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 msgr2=0x7fe9380b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 0x7fe9380b6120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x558ffdaa19b0 tx=0x7fe93c02a410 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 msgr2=0x7fe94c198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009a20 tx=0x7fe93c004270 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 shutdown_connections 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe94c108e60 0x7fe94c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9380b3c60 0x7fe9380b6120 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe94c1084e0 0x7fe94c199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 --2- 172.21.15.107:0/3731913981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe94c102ad0 0x7fe94c198ec0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 >> 172.21.15.107:0/3731913981 conn(0x7fe94c0fe540 msgr2=0x7fe94c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 shutdown_connections 2024-08-20T06:37:06.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:06.521+0000 7fe95142d640 1 -- 172.21.15.107:0/3731913981 wait complete. 2024-08-20T06:37:06.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:06 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:37:06.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:06 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:06.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:06 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:37:06.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:06 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:06 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:37:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:06 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:06 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:37:06.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:06 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:07.348 INFO:tasks.cephfs.filesystem:scrub status for tag:813f0697-6723-4f45-913a-b85bca0595cd - {'path': '/', 'tag': '813f0697-6723-4f45-913a-b85bca0595cd', 'options': 'recursive,force'} 2024-08-20T06:37:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:07 smithi193 ceph-mon[21944]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:07 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:37:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:07 smithi107 ceph-mon[18495]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:07 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:37:07.662 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:07 smithi183 ceph-mon[22204]: pgmap v335: 97 pgs: 97 active+clean; 650 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:07.663 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:07 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:37:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: pgmap v336: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:37:09.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: pgmap v336: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-20T06:37:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:37:09.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: pgmap v336: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:37:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:37:09.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:37:10.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:10 smithi107 ceph-mon[18495]: pgmap v337: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:10.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:10 smithi183 ceph-mon[22204]: pgmap v337: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:10 smithi193 ceph-mon[21944]: pgmap v337: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T06:37:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:12 smithi183 ceph-mon[22204]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-08-20T06:37:12.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:12 smithi193 ceph-mon[21944]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-08-20T06:37:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:12 smithi107 ceph-mon[18495]: pgmap v338: 97 pgs: 97 active+clean; 656 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-08-20T06:37:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:14 smithi183 ceph-mon[22204]: pgmap v339: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-20T06:37:14.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:14 smithi193 ceph-mon[21944]: pgmap v339: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-20T06:37:14.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:14 smithi107 ceph-mon[18495]: pgmap v339: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.5 KiB/s wr, 8 op/s 2024-08-20T06:37:15.705 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:15 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:37:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:15 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:37:15.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:15 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:37:16.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:16 smithi183 ceph-mon[22204]: pgmap v340: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T06:37:16.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:16 smithi193 ceph-mon[21944]: pgmap v340: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T06:37:16.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:16 smithi107 ceph-mon[18495]: pgmap v340: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T06:37:18.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:18 smithi183 ceph-mon[22204]: pgmap v341: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:18.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:18 smithi193 ceph-mon[21944]: pgmap v341: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:18.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:18 smithi107 ceph-mon[18495]: pgmap v341: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-08-20T06:37:20.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:20 smithi183 ceph-mon[22204]: pgmap v342: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-20T06:37:20.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:20 smithi193 ceph-mon[21944]: pgmap v342: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-20T06:37:20.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:20 smithi107 ceph-mon[18495]: pgmap v342: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 682 B/s wr, 6 op/s 2024-08-20T06:37:22.663 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:22 smithi107 ceph-mon[18495]: pgmap v343: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T06:37:22.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:22 smithi183 ceph-mon[22204]: pgmap v343: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T06:37:22.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:22 smithi193 ceph-mon[21944]: pgmap v343: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T06:37:24.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:24 smithi183 ceph-mon[22204]: pgmap v344: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T06:37:24.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:24 smithi193 ceph-mon[21944]: pgmap v344: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T06:37:24.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:24 smithi107 ceph-mon[18495]: pgmap v344: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T06:37:26.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:26 smithi183 ceph-mon[22204]: pgmap v345: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:37:26.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:26 smithi193 ceph-mon[21944]: pgmap v345: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:37:26.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:26 smithi107 ceph-mon[18495]: pgmap v345: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-08-20T06:37:28.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:28 smithi183 ceph-mon[22204]: pgmap v346: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-20T06:37:28.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:28 smithi193 ceph-mon[21944]: pgmap v346: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-20T06:37:28.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:28 smithi107 ceph-mon[18495]: pgmap v346: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 85 B/s wr, 6 op/s 2024-08-20T06:37:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:30 smithi183 ceph-mon[22204]: pgmap v347: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:30.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:30 smithi193 ceph-mon[21944]: pgmap v347: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:30.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:30 smithi107 ceph-mon[18495]: pgmap v347: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:32.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:32 smithi183 ceph-mon[22204]: pgmap v348: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:32.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:32 smithi193 ceph-mon[21944]: pgmap v348: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:32.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:32 smithi107 ceph-mon[18495]: pgmap v348: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:34.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:34 smithi183 ceph-mon[22204]: pgmap v349: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:37:34.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:34 smithi193 ceph-mon[21944]: pgmap v349: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:37:34.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:34 smithi107 ceph-mon[18495]: pgmap v349: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-08-20T06:37:36.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:36 smithi183 ceph-mon[22204]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:36.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:36 smithi193 ceph-mon[21944]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:36.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:36 smithi107 ceph-mon[18495]: pgmap v350: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:37.349 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:37:37.613 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:38.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 -- 172.21.15.107:0/2946551199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c107d20 msgr2=0x7f815c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 --2- 172.21.15.107:0/2946551199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c107d20 0x7f815c10e520 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8154009860 tx=0x7f815402a340 comp rx=0 tx=0).stop 2024-08-20T06:37:38.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 -- 172.21.15.107:0/2946551199 shutdown_connections 2024-08-20T06:37:38.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 --2- 172.21.15.107:0/2946551199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c107d20 0x7f815c10e520 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 --2- 172.21.15.107:0/2946551199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 --2- 172.21.15.107:0/2946551199 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c101990 0x7f815c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 -- 172.21.15.107:0/2946551199 >> 172.21.15.107:0/2946551199 conn(0x7f815c0fd400 msgr2=0x7f815c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 -- 172.21.15.107:0/2946551199 shutdown_connections 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.107+0000 7f8163e32640 1 -- 172.21.15.107:0/2946551199 wait complete. 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 Processor -- start 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 -- start start 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c107d20 0x7f815c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c112980 con 0x7f815c1073a0 2024-08-20T06:37:38.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f815c112800 con 0x7f815c107d20 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8163e32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f815c112b00 con 0x7f815c101990 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8161bce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8161bce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:58682/0 (socket says 172.21.15.107:58682) 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f8161bce640 1 -- 172.21.15.107:0/1231250785 learned_addr learned my addr 172.21.15.107:0/1231250785 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.108+0000 7f81623cf640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c107d20 0x7f815c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f81613cd640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 -- 172.21.15.107:0/1231250785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c107d20 msgr2=0x7f815c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c107d20 0x7f815c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 -- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 msgr2=0x7f815c1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 -- 172.21.15.107:0/1231250785 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815c19ef20 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f81613cd640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8161bce640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f815800eb90 tx=0x7f815800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8152ffd640 1 -- 172.21.15.107:0/1231250785 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158004d20 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8152ffd640 1 -- 172.21.15.107:0/1231250785 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f815800ee90 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8152ffd640 1 -- 172.21.15.107:0/1231250785 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158010690 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f815c19f210 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.109+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f815c10ebe0 con 0x7f815c101990 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 Processor -- start 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 -- start start 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c72100e0 0x5619c72124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619c717a8e0 con 0x5619c7203830 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5619c7169de0 con 0x5619c72100e0 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f816622b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619c716de20 con 0x5619c7204150 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36214/0 (socket says 172.21.15.107:36214) 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f81623cf640 1 -- 172.21.15.107:0/2066900136 learned_addr learned my addr 172.21.15.107:0/2066900136 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.113+0000 7f8152ffd640 1 -- 172.21.15.107:0/1231250785 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f8158005020 con 0x7f815c101990 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81613cd640 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c72100e0 0x5619c72124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 -- 172.21.15.107:0/2066900136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 msgr2=0x5619c720fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 -- 172.21.15.107:0/2066900136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c72100e0 msgr2=0x5619c72124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c72100e0 0x5619c72124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 -- 172.21.15.107:0/2066900136 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c6ec9650 con 0x5619c7203830 2024-08-20T06:37:38.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f81623cf640 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8154009830 tx=0x7f8154009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813af61e8702244 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f8147fff640 1 -- 172.21.15.107:0/2066900136 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037040 con 0x5619c7203830 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f8147fff640 1 -- 172.21.15.107:0/2066900136 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8154030e70 con 0x5619c7203830 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.114+0000 7f8147fff640 1 -- 172.21.15.107:0/2066900136 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81540052d0 con 0x5619c7203830 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f8152ffd640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 0x7f81480b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f8161bce640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 0x7f81480b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f8152ffd640 1 -- 172.21.15.107:0/1231250785 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f81580e4410 con 0x7f815c101990 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 -- 172.21.15.107:0/2066900136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 msgr2=0x5619c7203c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8154009830 tx=0x7f8154009510 comp rx=0 tx=0).stop 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 -- 172.21.15.107:0/2066900136 shutdown_connections 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c72100e0 0x5619c72124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 --2- 172.21.15.107:0/2066900136 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c7203c10 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 -- 172.21.15.107:0/2066900136 >> 172.21.15.107:0/2066900136 conn(0x5619c7201660 msgr2=0x5619c7202800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 -- 172.21.15.107:0/2066900136 shutdown_connections 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.115+0000 7f816622b740 1 -- 172.21.15.107:0/2066900136 wait complete. 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 -- start start 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 Processor -- start 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f8161bce640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 0x7f81480b6220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81580a7240 tx=0x7f8158070030 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7204150 0x5619c720d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c720db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619c6fcc700 con 0x5619c7203830 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5619c70f07c0 con 0x5619c7204150 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.116+0000 7f816622b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619c7212d30 con 0x5619c72100e0 2024-08-20T06:37:38.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36218/0 (socket says 172.21.15.107:36218) 2024-08-20T06:37:38.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 -- 172.21.15.107:0/3642191014 learned_addr learned my addr 172.21.15.107:0/3642191014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 -- 172.21.15.107:0/3642191014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 msgr2=0x5619c720db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c720db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 -- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7204150 msgr2=0x5619c720d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81613cd640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7204150 0x5619c720d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f816622b740 10 client.0 _ll_register_callbacks cb 0x5619c71636b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7204150 0x5619c720d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 -- 172.21.15.107:0/3642191014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c7224920 con 0x5619c7203830 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619c7119100 con 0x5619c7203830 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81623cf640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8154009780 tx=0x7f81540303d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6cea304952a3c19 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.117+0000 7f81457fa640 10 client.15450 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f81457fa640 1 -- 172.21.15.107:0/3642191014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154037030 con 0x5619c7203830 2024-08-20T06:37:38.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f81457fa640 1 -- 172.21.15.107:0/3642191014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8154004300 con 0x5619c7203830 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f81457fa640 1 -- 172.21.15.107:0/3642191014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8154005050 con 0x5619c7203830 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 10 client.15450 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5619c72271f0 con 0x5619c7203830 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f81457fa640 1 -- 172.21.15.107:0/3642191014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f81540051f0 con 0x5619c7203830 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 10 client.15450 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 10 client.15450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 20 client.15450 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c7228540 0x5619c722a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 4 client.15450 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.118+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619c717c830 con 0x5619c7228540 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.119+0000 7f81613cd640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c7228540 0x5619c722a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.119+0000 7f81613cd640 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c7228540 0x5619c722a930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55d596469ba4e285 server_cookie=f9daf8134640ff5 in_seq=0 out_seq=0 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.119+0000 7f81457fa640 10 client.15450 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:38.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f81457fa640 1 -- 172.21.15.107:0/3642191014 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619c717c830 con 0x5619c7228540 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f81457fa640 10 client.15450 handle_command_reply: tid=0 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 client.15450 shutdown 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 msgr2=0x5619c720d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8154009780 tx=0x7f81540303d0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 shutdown_connections 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c7228540 0x5619c722a930 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c720db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7204150 0x5619c720d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 --2- 172.21.15.107:0/3642191014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7203830 0x5619c720d080 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 >> 172.21.15.107:0/3642191014 conn(0x5619c7201660 msgr2=0x5619c7207200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.120+0000 7f81457fa640 0 client.15450 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.121+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 shutdown_connections 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.121+0000 7f816622b740 1 -- 172.21.15.107:0/3642191014 wait complete. 2024-08-20T06:37:38.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.121+0000 7f816622b740 20 client.15450 trim_cache size 0 max 16384 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 Processor -- start 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 -- start start 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c720c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1ae5a0 con 0x5619c72100e0 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1abe70 con 0x5619c7203830 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f816622b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1a9ca0 con 0x5619c7204150 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f81623cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c720c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f81613cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f81623cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36222/0 (socket says 172.21.15.107:36222) 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.138+0000 7f81623cf640 1 -- 172.21.15.107:0/3354500262 learned_addr learned my addr 172.21.15.107:0/3354500262 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:38.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 -- 172.21.15.107:0/3354500262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 msgr2=0x5619c720c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c720c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 -- 172.21.15.107:0/3354500262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 msgr2=0x5619c722d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 -- 172.21.15.107:0/3354500262 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c720c020 con 0x5619c7204150 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81623cf640 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81613cd640 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720d760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5619c7212d60 tx=0x7f814c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3053403911ee7058 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81477fe640 1 -- 172.21.15.107:0/3354500262 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01f070 con 0x5619c7204150 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81477fe640 1 -- 172.21.15.107:0/3354500262 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f814c0035f0 con 0x5619c7204150 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.139+0000 7f81477fe640 1 -- 172.21.15.107:0/3354500262 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01b520 con 0x5619c7204150 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- 172.21.15.107:0/3354500262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 msgr2=0x5619c720d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720d760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x5619c7212d60 tx=0x7f814c009260 comp rx=0 tx=0).stop 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- 172.21.15.107:0/3354500262 shutdown_connections 2024-08-20T06:37:38.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c72100e0 0x5619c722d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c7204150 0x5619c720d760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 --2- 172.21.15.107:0/3354500262 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c720c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- 172.21.15.107:0/3354500262 >> 172.21.15.107:0/3354500262 conn(0x5619c7201660 msgr2=0x5619c720d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- 172.21.15.107:0/3354500262 shutdown_connections 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- 172.21.15.107:0/3354500262 wait complete. 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.140+0000 7f816622b740 1 -- start start 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 Processor -- start 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c722dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c722b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1aa330 con 0x5619c7204150 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1ab7a0 con 0x5619c7203830 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f815c1aaa00 con 0x5619c72100e0 2024-08-20T06:37:38.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f816622b740 10 client.0 _ll_register_callbacks cb 0x7f81580093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f81613cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f81613cd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36236/0 (socket says 172.21.15.107:36236) 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f81613cd640 1 -- 172.21.15.107:0/2683092040 learned_addr learned my addr 172.21.15.107:0/2683092040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.141+0000 7f81623cf640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c722b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81623cf640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c722dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 -- 172.21.15.107:0/2683092040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 msgr2=0x5619c722b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c722b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 -- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 msgr2=0x5619c722dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c722dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 -- 172.21.15.107:0/2683092040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c7291e80 con 0x5619c7204150 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81623cf640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c722dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81613cd640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f814c005d30 tx=0x7f814c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ac5dd87fdb42d8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:38.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81457fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01f040 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f814c004020 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c0035f0 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f814c004220 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f816622b740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.142+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619c7292fe0 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f814c0043c0 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f814c00f040 con 0x5619c7204150 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 10 client.15456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 20 client.15456 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c72935a0 0x5619c72959c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 4 client.15456 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:37:38.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f814c0035f0 con 0x5619c72935a0 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.143+0000 7f81623cf640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c72935a0 0x5619c72959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f81457fa640 10 client.15456 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f81623cf640 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c72935a0 0x5619c72959c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec30725c1089e700 server_cookie=eca40838d7a35059 in_seq=0 out_seq=0 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f81457fa640 1 -- 172.21.15.107:0/2683092040 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f814c0035f0 con 0x5619c72935a0 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f81457fa640 10 client.15456 handle_command_reply: tid=0 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f816622b740 1 client.15456 shutdown 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 msgr2=0x5619c722b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.144+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f814c005d30 tx=0x7f814c005d60 comp rx=0 tx=0).stop 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 shutdown_connections 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619c72935a0 0x5619c72959c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619c72100e0 0x5619c722b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619c7204150 0x5619c722b1f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 --2- 172.21.15.107:0/2683092040 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619c7203830 0x5619c722dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 >> 172.21.15.107:0/2683092040 conn(0x5619c7201660 msgr2=0x5619c7207200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f81457fa640 0 client.15456 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 shutdown_connections 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 1 -- 172.21.15.107:0/2683092040 wait complete. 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f816622b740 20 client.15456 trim_cache size 0 max 16384 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 msgr2=0x7f81480b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 0x7f81480b6220 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f81580a7240 tx=0x7f8158070030 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 msgr2=0x7f815c198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.145+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f815800eb90 tx=0x7f815800c6a0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 shutdown_connections 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f815c107d20 0x7f815c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f815c1073a0 0x7f815c1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f81480b3d60 0x7f81480b6220 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 --2- 172.21.15.107:0/1231250785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f815c101990 0x7f815c198e70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 >> 172.21.15.107:0/1231250785 conn(0x7f815c0fd400 msgr2=0x7f815c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:38.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 shutdown_connections 2024-08-20T06:37:38.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:38.146+0000 7f8163e32640 1 -- 172.21.15.107:0/1231250785 wait complete. 2024-08-20T06:37:38.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:38 smithi183 ceph-mon[22204]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:37:38.778 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:38 smithi107 ceph-mon[18495]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:37:38.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:38 smithi193 ceph-mon[21944]: pgmap v351: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-08-20T06:37:38.947 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:37:38.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:37:39.226 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1753660808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc102aa0 msgr2=0x7fe7cc102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1753660808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc102e80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0009900 tx=0x7fe7c002a410 comp rx=0 tx=0).stop 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1753660808 shutdown_connections 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1753660808 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc108e30 0x7fe7cc10f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1753660808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1753660808 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc102e80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1753660808 >> 172.21.15.107:0/1753660808 conn(0x7fe7cc0fe550 msgr2=0x7fe7cc100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:39.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1753660808 shutdown_connections 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.756+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1753660808 wait complete. 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.757+0000 7fe7d2c0b640 1 Processor -- start 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.757+0000 7fe7d2c0b640 1 -- start start 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc108e30 0x7fe7cc06d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc1129f0 con 0x7fe7cc1084b0 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc112870 con 0x7fe7cc102aa0 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d2c0b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe7cc112b70 con 0x7fe7cc108e30 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7cbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc074a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36260/0 (socket says 172.21.15.107:36260) 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7cbfff640 1 -- 172.21.15.107:0/1251610031 learned_addr learned my addr 172.21.15.107:0/1251610031 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:39.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d11a8640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc108e30 0x7fe7cc06d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 -- 172.21.15.107:0/1251610031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc108e30 msgr2=0x7fe7cc06d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc108e30 0x7fe7cc06d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 -- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 msgr2=0x7fe7cc074a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc074a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 -- 172.21.15.107:0/1251610031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7cc06e170 con 0x7fe7cc102aa0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.758+0000 7fe7d09a7640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc074500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a930 tx=0x7fe7c0003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.759+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c003f070 con 0x7fe7cc102aa0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.759+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7cc06e400 con 0x7fe7cc102aa0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.759+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe7c003baf0 con 0x7fe7cc102aa0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.759+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0035030 con 0x7fe7cc102aa0 2024-08-20T06:37:39.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.759+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7cc073390 con 0x7fe7cc102aa0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.760+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7cc102e80 con 0x7fe7cc102aa0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.762+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7fe7c0042070 con 0x7fe7cc102aa0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.763+0000 7fe7c9ffb640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 0x7fe7bc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.763+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe7c0104920 con 0x7fe7cc102aa0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.763+0000 7fe7cbfff640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 0x7fe7bc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.764+0000 7fe7cbfff640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 0x7fe7bc0b6130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc101b40 tx=0x7fe7b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:39.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.765+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe7c00cb780 con 0x7fe7cc102aa0 2024-08-20T06:37:39.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.991+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7cc06e5d0 con 0x7fe7cc102aa0 2024-08-20T06:37:39.993 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:37:39.993 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"btime":"2024-08-20T06:36:56:241712+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:36:56.009554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:37:39.994 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.992+0000 7fe7c9ffb640 1 -- 172.21.15.107:0/1251610031 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7fe7c00cfcd0 con 0x7fe7cc102aa0 2024-08-20T06:37:39.994 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.995+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 msgr2=0x7fe7bc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.995+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 0x7fe7bc0b6130 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc101b40 tx=0x7fe7b8005f90 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 msgr2=0x7fe7cc074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc074500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7c002a930 tx=0x7fe7c0003b90 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 shutdown_connections 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe7bc0b3c70 0x7fe7bc0b6130 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7cc108e30 0x7fe7cc06d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe7cc1084b0 0x7fe7cc074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 --2- 172.21.15.107:0/1251610031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe7cc102aa0 0x7fe7cc074500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 >> 172.21.15.107:0/1251610031 conn(0x7fe7cc0fe550 msgr2=0x7fe7cc100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:39.997 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 shutdown_connections 2024-08-20T06:37:39.998 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:39.996+0000 7fe7d2c0b640 1 -- 172.21.15.107:0/1251610031 wait complete. 2024-08-20T06:37:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:40 smithi107 ceph-mon[18495]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:40 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1251610031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:40.673 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:37:40.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:40 smithi183 ceph-mon[22204]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:40.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:40 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1251610031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:40.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:40 smithi193 ceph-mon[21944]: pgmap v352: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-08-20T06:37:40.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:40 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1251610031' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:37:40.959 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.526+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/401330148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a141015b0 msgr2=0x7f3a14101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.526+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/401330148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a141015b0 0x7f3a14101990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3a10009900 tx=0x7f3a1002a410 comp rx=0 tx=0).stop 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/401330148 shutdown_connections 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/401330148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a1410f160 0x7f3a14111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/401330148 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a14101ed0 0x7f3a1410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/401330148 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a141015b0 0x7f3a14101990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/401330148 >> 172.21.15.107:0/401330148 conn(0x7f3a140fd400 msgr2=0x7f3a140ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/401330148 shutdown_connections 2024-08-20T06:37:41.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/401330148 wait complete. 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 Processor -- start 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.527+0000 7f3a1c0a0640 1 -- start start 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a141015b0 0x7f3a1419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 0x7f3a1419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114bc0 con 0x7f3a1410f160 2024-08-20T06:37:41.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114a40 con 0x7f3a141015b0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1c0a0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a14114d40 con 0x7f3a14101ed0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36266/0 (socket says 172.21.15.107:36266) 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 -- 172.21.15.107:0/445916096 learned_addr learned my addr 172.21.15.107:0/445916096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1963b640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 0x7f3a1419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a141015b0 0x7f3a1419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 -- 172.21.15.107:0/445916096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 msgr2=0x7f3a1419b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 0x7f3a1419b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 -- 172.21.15.107:0/445916096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a141015b0 msgr2=0x7f3a1419b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a141015b0 0x7f3a1419b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1a63d640 1 -- 172.21.15.107:0/445916096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a141a1180 con 0x7f3a1410f160 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.528+0000 7f3a1963b640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 0x7f3a1419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00eb40 tx=0x7f3a0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a0affd640 1 -- 172.21.15.107:0/445916096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c004b20 con 0x7f3a1410f160 2024-08-20T06:37:41.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a141a1470 con 0x7f3a1410f160 2024-08-20T06:37:41.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a0affd640 1 -- 172.21.15.107:0/445916096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a0c004cc0 con 0x7f3a1410f160 2024-08-20T06:37:41.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a0affd640 1 -- 172.21.15.107:0/445916096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c010690 con 0x7f3a1410f160 2024-08-20T06:37:41.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.529+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a141037c0 con 0x7f3a1410f160 2024-08-20T06:37:41.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.532+0000 7f3a1e499740 1 Processor -- start 2024-08-20T06:37:41.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a0affd640 1 -- 172.21.15.107:0/445916096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f3a0c005540 con 0x7f3a1410f160 2024-08-20T06:37:41.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 -- start start 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d2fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d3bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564909ca6930 con 0x564909d3c130 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564909c95e30 con 0x564909d2f880 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1e499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564909c99e70 con 0x564909d301a0 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a19e3c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d3bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36276/0 (socket says 172.21.15.107:36276) 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.533+0000 7f3a1963b640 1 -- 172.21.15.107:0/4120372028 learned_addr learned my addr 172.21.15.107:0/4120372028 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d2fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 -- 172.21.15.107:0/4120372028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 msgr2=0x564909d3bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d3bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 -- 172.21.15.107:0/4120372028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 msgr2=0x564909d2fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d2fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 -- 172.21.15.107:0/4120372028 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649099f5650 con 0x564909d3c130 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a0affd640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 0x7f3a000be930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 0x7f3a000be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a1963b640 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a04014a90 tx=0x7f3a0400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95af19c3deec1f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f3a0affd640 1 -- 172.21.15.107:0/445916096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3a0c0e44e0 con 0x7f3a1410f160 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f39fffff640 1 -- 172.21.15.107:0/4120372028 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04004030 con 0x564909d3c130 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.534+0000 7f39fffff640 1 -- 172.21.15.107:0/4120372028 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a04005020 con 0x564909d3c130 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f39fffff640 1 -- 172.21.15.107:0/4120372028 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400fa80 con 0x564909d3c130 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 -- 172.21.15.107:0/4120372028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 msgr2=0x564909d3e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3a04014a90 tx=0x7f3a0400b5b0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 0x7f3a000be930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a10006010 tx=0x7f3a10005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 -- 172.21.15.107:0/4120372028 shutdown_connections 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d3c130 0x564909d3e520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d3bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 --2- 172.21.15.107:0/4120372028 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d2fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.535+0000 7f3a1e499740 1 -- 172.21.15.107:0/4120372028 >> 172.21.15.107:0/4120372028 conn(0x564909d2d6b0 msgr2=0x564909d2e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.536+0000 7f3a1e499740 1 -- 172.21.15.107:0/4120372028 shutdown_connections 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.536+0000 7f3a1e499740 1 -- 172.21.15.107:0/4120372028 wait complete. 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.536+0000 7f3a1e499740 1 -- start start 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.536+0000 7f3a1e499740 1 Processor -- start 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 0x564909d39b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x564909af8700 con 0x564909d301a0 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x564909c1c810 con 0x564909d2f880 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x564909d3ee30 con 0x564909d3c130 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1e499740 10 client.0 _ll_register_callbacks cb 0x564909c8f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36284/0 (socket says 172.21.15.107:36284) 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 -- 172.21.15.107:0/3202124071 learned_addr learned my addr 172.21.15.107:0/3202124071 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 0x564909d39b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 -- 172.21.15.107:0/3202124071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 msgr2=0x564909d39b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 0x564909d39b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 -- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 msgr2=0x564909d390c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d390c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 -- 172.21.15.107:0/3202124071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564909d481e0 con 0x564909d301a0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 0x564909d39b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f3a1963b640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005cb0 tx=0x7f3a0400b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76534344cf7f8290 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.537+0000 7f39fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04005220 con 0x564909d301a0 2024-08-20T06:37:41.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3a040053c0 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564909c45150 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a0400f820 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 10 client.15474 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564909d4ab90 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x564909c45150 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a04004a10 con 0x564909d301a0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 10 client.15474 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 10 client.15474 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 20 client.15474 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909d4e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 4 client.15474 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.538+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564909ca8880 con 0x564909d4bff0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.539+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909d4e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.539+0000 7f39fd7fa640 10 client.15474 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:37:41.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.539+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909d4e3e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1815d7d7a3c446cc server_cookie=df902beee62b75e5 in_seq=0 out_seq=0 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f39fd7fa640 1 -- 172.21.15.107:0/3202124071 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564909ca8880 con 0x564909d4bff0 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f39fd7fa640 10 client.15474 handle_command_reply: tid=0 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 client.15474 shutdown 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 msgr2=0x564909d39600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3a04005cb0 tx=0x7f3a0400b2e0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 shutdown_connections 2024-08-20T06:37:41.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d3c130 0x564909d39b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d39600 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909d4e3e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 --2- 172.21.15.107:0/3202124071 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d2f880 0x564909d390c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f39fd7fa640 0 client.15474 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.540+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 >> 172.21.15.107:0/3202124071 conn(0x564909d2d6b0 msgr2=0x564909d33220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.541+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 shutdown_connections 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.541+0000 7f3a1e499740 1 -- 172.21.15.107:0/3202124071 wait complete. 2024-08-20T06:37:41.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.541+0000 7f3a1e499740 20 client.15474 trim_cache size 0 max 16384 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 Processor -- start 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 -- start start 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 0x564909d338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d54130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ae770 con 0x564909d301a0 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ac040 con 0x564909d3c130 2024-08-20T06:37:41.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1e499740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141a9e70 con 0x564909d2f880 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1963b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36298/0 (socket says 172.21.15.107:36298) 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.559+0000 7f3a1963b640 1 -- 172.21.15.107:0/2334721035 learned_addr learned my addr 172.21.15.107:0/2334721035 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 0x564909d338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d54130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 -- 172.21.15.107:0/2334721035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 msgr2=0x564909d338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 0x564909d338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 -- 172.21.15.107:0/2334721035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 msgr2=0x564909d54130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d54130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 -- 172.21.15.107:0/2334721035 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564909d28830 con 0x564909d301a0 2024-08-20T06:37:41.562 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 0x564909d338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d54130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f3a1963b640 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b5f0 tx=0x7f3a0400bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18436fd805ba36fa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f39ff7fe640 1 -- 172.21.15.107:0/2334721035 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400f9c0 con 0x564909d301a0 2024-08-20T06:37:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f39ff7fe640 1 -- 172.21.15.107:0/2334721035 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a04024030 con 0x564909d301a0 2024-08-20T06:37:41.563 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.560+0000 7f39ff7fe640 1 -- 172.21.15.107:0/2334721035 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04020ae0 con 0x564909d301a0 2024-08-20T06:37:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 -- 172.21.15.107:0/2334721035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 msgr2=0x564909d53bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f3a0400b5f0 tx=0x7f3a0400bb70 comp rx=0 tx=0).stop 2024-08-20T06:37:41.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 -- 172.21.15.107:0/2334721035 shutdown_connections 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d54130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d301a0 0x564909d53bf0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 --2- 172.21.15.107:0/2334721035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d2f880 0x564909d338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 -- 172.21.15.107:0/2334721035 >> 172.21.15.107:0/2334721035 conn(0x564909d2d6b0 msgr2=0x564909d3af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 -- 172.21.15.107:0/2334721035 shutdown_connections 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.561+0000 7f3a1e499740 1 -- 172.21.15.107:0/2334721035 wait complete. 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.562+0000 7f3a1e499740 1 -- start start 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.562+0000 7f3a1e499740 1 Processor -- start 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.562+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d41480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a19e3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a19e3c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36312/0 (socket says 172.21.15.107:36312) 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a19e3c640 1 -- 172.21.15.107:0/1234884505 learned_addr learned my addr 172.21.15.107:0/1234884505 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:41.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141aa500 con 0x564909d2f880 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141ab970 con 0x564909d3c130 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3a141aabd0 con 0x564909d301a0 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d41480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1e499740 10 client.0 _ll_register_callbacks cb 0x7f3a04014470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1a63d640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 -- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 msgr2=0x564909d419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 -- 172.21.15.107:0/1234884505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 msgr2=0x564909d40f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564909db4c40 con 0x564909d301a0 2024-08-20T06:37:41.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.563+0000 7f3a1963b640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d41480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3a04014a90 tx=0x7f3a0400feb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89facefb3a6c9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04020450 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a04024030 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3a04020a70 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3a04020c70 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 10 client.24964 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564909db5e50 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3a04024030 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3a04005480 con 0x564909d301a0 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 10 client.24964 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 10 client.24964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 20 client.24964 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909db8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:41.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 4 client.24964 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.564+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564909ca4320 con 0x564909d4bff0 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.565+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909db8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.565+0000 7f39fd7fa640 10 client.24964 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.565+0000 7f3a19e3c640 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909db8520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=89225931b7947504 server_cookie=c7712af00fb29671 in_seq=0 out_seq=0 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f39fd7fa640 1 -- 172.21.15.107:0/1234884505 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564909ca4320 con 0x564909d4bff0 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f39fd7fa640 10 client.24964 handle_command_reply: tid=0 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 client.24964 shutdown 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 msgr2=0x564909d41480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d41480 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3a04014a90 tx=0x7f3a0400feb0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 shutdown_connections 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x564909d4bff0 0x564909db8520 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x564909d3c130 0x564909d419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564909d301a0 0x564909d41480 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f39fd7fa640 0 client.24964 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 --2- 172.21.15.107:0/1234884505 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x564909d2f880 0x564909d40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 >> 172.21.15.107:0/1234884505 conn(0x564909d2d6b0 msgr2=0x564909d46f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 shutdown_connections 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 1 -- 172.21.15.107:0/1234884505 wait complete. 2024-08-20T06:37:41.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.566+0000 7f3a1e499740 20 client.24964 trim_cache size 0 max 16384 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 msgr2=0x7f3a000be930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 0x7f3a000be930 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3a10006010 tx=0x7f3a10005e80 comp rx=0 tx=0).stop 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 msgr2=0x7f3a141a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c00eb40 tx=0x7f3a0c00c2d0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 shutdown_connections 2024-08-20T06:37:41.572 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a1410f160 0x7f3a141a09b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3a000bc470 0x7f3a000be930 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a14101ed0 0x7f3a1419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 --2- 172.21.15.107:0/445916096 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3a141015b0 0x7f3a1419b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 >> 172.21.15.107:0/445916096 conn(0x7f3a140fd400 msgr2=0x7f3a140ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 shutdown_connections 2024-08-20T06:37:41.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:41.567+0000 7f3a1c0a0640 1 -- 172.21.15.107:0/445916096 wait complete. 2024-08-20T06:37:42.295 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:37:42.434 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:42 smithi107 ceph-mon[18495]: pgmap v353: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:42.595 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:42.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:42 smithi183 ceph-mon[22204]: pgmap v353: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:42.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:42 smithi193 ceph-mon[21944]: pgmap v353: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 -- 172.21.15.107:0/3884242719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 msgr2=0x7f674c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 --2- 172.21.15.107:0/3884242719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c10e520 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6744009ca0 tx=0x7f674402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 -- 172.21.15.107:0/3884242719 shutdown_connections 2024-08-20T06:37:43.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 --2- 172.21.15.107:0/3884242719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c10e520 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 --2- 172.21.15.107:0/3884242719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c1073a0 0x7f674c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 --2- 172.21.15.107:0/3884242719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f674c101990 0x7f674c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 -- 172.21.15.107:0/3884242719 >> 172.21.15.107:0/3884242719 conn(0x7f674c0fd400 msgr2=0x7f674c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 -- 172.21.15.107:0/3884242719 shutdown_connections 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.041+0000 7f6750d60640 1 -- 172.21.15.107:0/3884242719 wait complete. 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 Processor -- start 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 -- start start 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c101990 0x7f674c19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f674c1073a0 0x7f674c19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f674c1118e0 con 0x7f674c107d20 2024-08-20T06:37:43.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f674c111760 con 0x7f674c101990 2024-08-20T06:37:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f6750d60640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f674c111a60 con 0x7f674c1073a0 2024-08-20T06:37:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36338/0 (socket says 172.21.15.107:36338) 2024-08-20T06:37:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f674ad9d640 1 -- 172.21.15.107:0/3950569078 learned_addr learned my addr 172.21.15.107:0/3950569078 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:43.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.042+0000 7f674a59c640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c101990 0x7f674c19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 -- 172.21.15.107:0/3950569078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f674c1073a0 msgr2=0x7f674c19b620 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f674c1073a0 0x7f674c19b620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 -- 172.21.15.107:0/3950569078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c101990 msgr2=0x7f674c19b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c101990 0x7f674c19b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 -- 172.21.15.107:0/3950569078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c1a1190 con 0x7f674c107d20 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6744031040 tx=0x7f6744003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.043+0000 7f673b7fe640 1 -- 172.21.15.107:0/3950569078 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744032c90 con 0x7f674c107d20 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.044+0000 7f673b7fe640 1 -- 172.21.15.107:0/3950569078 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6744032e30 con 0x7f674c107d20 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.044+0000 7f673b7fe640 1 -- 172.21.15.107:0/3950569078 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6744030b10 con 0x7f674c107d20 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.044+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c1a1420 con 0x7f674c107d20 2024-08-20T06:37:43.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.044+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c1a7b40 con 0x7f674c107d20 2024-08-20T06:37:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f673b7fe640 1 -- 172.21.15.107:0/3950569078 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f6744030cb0 con 0x7f674c107d20 2024-08-20T06:37:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 Processor -- start 2024-08-20T06:37:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 -- start start 2024-08-20T06:37:43.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bd4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bd4940 0x560ef5bd6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560ef5b36930 con 0x560ef5bc8090 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ef5b25e30 con 0x560ef5bc89b0 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6753159740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ef5b29e70 con 0x560ef5bd4940 2024-08-20T06:37:43.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6749d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6749d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36354/0 (socket says 172.21.15.107:36354) 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.048+0000 7f6749d9b640 1 -- 172.21.15.107:0/948427690 learned_addr learned my addr 172.21.15.107:0/948427690 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bd4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 -- 172.21.15.107:0/948427690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bd4940 msgr2=0x560ef5bd6d30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bd4940 0x560ef5bd6d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 -- 172.21.15.107:0/948427690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 msgr2=0x560ef5bc8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 -- 172.21.15.107:0/948427690 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef5885650 con 0x560ef5bc89b0 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f674a59c640 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bd4400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f673c004630 tx=0x7f673c00bb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9fc2a806cca8c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.049+0000 7f6749d9b640 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f673b7fe640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 0x7f67340b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f6738ff9640 1 -- 172.21.15.107:0/948427690 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c003bd0 con 0x560ef5bc89b0 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f6738ff9640 1 -- 172.21.15.107:0/948427690 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f673c015070 con 0x560ef5bc89b0 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f6738ff9640 1 -- 172.21.15.107:0/948427690 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c004c60 con 0x560ef5bc89b0 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f6749d9b640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 0x7f67340b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.050+0000 7f673b7fe640 1 -- 172.21.15.107:0/3950569078 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f6744104d60 con 0x7f674c107d20 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- 172.21.15.107:0/948427690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 msgr2=0x560ef5bd4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bd4400 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f673c004630 tx=0x7f673c00bb20 comp rx=0 tx=0).stop 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6749d9b640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 0x7f67340b6270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6740009e70 tx=0x7f6740005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- 172.21.15.107:0/948427690 shutdown_connections 2024-08-20T06:37:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bd4940 0x560ef5bd6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bd4400 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 --2- 172.21.15.107:0/948427690 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bc8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.054 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- 172.21.15.107:0/948427690 >> 172.21.15.107:0/948427690 conn(0x560ef5bc5ec0 msgr2=0x560ef5bc7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- 172.21.15.107:0/948427690 shutdown_connections 2024-08-20T06:37:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- 172.21.15.107:0/948427690 wait complete. 2024-08-20T06:37:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 -- start start 2024-08-20T06:37:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.051+0000 7f6753159740 1 Processor -- start 2024-08-20T06:37:43.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bd4940 0x560ef5bd2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560ef5988700 con 0x560ef5bc8090 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560ef5aac810 con 0x560ef5bd4940 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560ef5bd75c0 con 0x560ef5bc89b0 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6753159740 10 client.0 _ll_register_callbacks cb 0x560ef5b1f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36368/0 (socket says 172.21.15.107:36368) 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 -- 172.21.15.107:0/1653413053 learned_addr learned my addr 172.21.15.107:0/1653413053 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674ad9d640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bd4940 0x560ef5bd2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6749d9b640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 -- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 msgr2=0x560ef5bd1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 -- 172.21.15.107:0/1653413053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bd4940 msgr2=0x560ef5bd2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bd4940 0x560ef5bd2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 -- 172.21.15.107:0/1653413053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef5bc40d0 con 0x560ef5bc8090 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6749d9b640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f674a59c640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f673c00bfd0 tx=0x7f673c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce676bec9647cb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.052+0000 7f6731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c028060 con 0x560ef5bc8090 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f673c023660 con 0x560ef5bc8090 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560ef5ad5150 con 0x560ef5bc8090 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f673c003f60 con 0x560ef5bc8090 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6753159740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:43.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ef5bd8b90 con 0x560ef5bc8090 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560ef5ad5150 con 0x560ef5bc8090 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.053+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f673c02d070 con 0x560ef5bc8090 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 10 client.15492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 20 client.15492 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5bd9f90 0x560ef5bdc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 4 client.15492 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef5b38880 con 0x560ef5bd9f90 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6749d9b640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5bd9f90 0x560ef5bdc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6731ffb640 10 client.15492 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:37:43.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.054+0000 7f6749d9b640 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5bd9f90 0x560ef5bdc3d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80a3a6d56daf8810 server_cookie=d74c934a6b85205e in_seq=0 out_seq=0 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6731ffb640 1 -- 172.21.15.107:0/1653413053 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ef5b38880 con 0x560ef5bd9f90 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6731ffb640 10 client.15492 handle_command_reply: tid=0 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 client.15492 shutdown 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 msgr2=0x560ef5bd18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f673c00bfd0 tx=0x7f673c004790 comp rx=0 tx=0).stop 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 shutdown_connections 2024-08-20T06:37:43.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5bd9f90 0x560ef5bdc3d0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bd4940 0x560ef5bd2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 --2- 172.21.15.107:0/1653413053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bc8090 0x560ef5bd18d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 >> 172.21.15.107:0/1653413053 conn(0x560ef5bc5ec0 msgr2=0x560ef5bcba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6731ffb640 0 client.15492 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 shutdown_connections 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.055+0000 7f6753159740 1 -- 172.21.15.107:0/1653413053 wait complete. 2024-08-20T06:37:43.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.056+0000 7f6753159740 20 client.15492 trim_cache size 0 max 16384 2024-08-20T06:37:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 Processor -- start 2024-08-20T06:37:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 -- start start 2024-08-20T06:37:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc8090 0x560ef5bd1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f672c010d90 con 0x560ef5bd4940 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f672c00e660 con 0x560ef5bc8090 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6753159740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f672c00c490 con 0x560ef5bc89b0 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36370/0 (socket says 172.21.15.107:36370) 2024-08-20T06:37:43.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 -- 172.21.15.107:0/3984681548 learned_addr learned my addr 172.21.15.107:0/3984681548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f6749d9b640 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674a59c640 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc8090 0x560ef5bd1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 -- 172.21.15.107:0/3984681548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 msgr2=0x560ef5bd2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 -- 172.21.15.107:0/3984681548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc8090 msgr2=0x560ef5bd1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc8090 0x560ef5bd1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.069+0000 7f674ad9d640 1 -- 172.21.15.107:0/3984681548 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef5bc7a70 con 0x560ef5bd4940 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.070+0000 7f674ad9d640 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67440c9f60 tx=0x7f67440c8c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2b844f5d4c5173 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.070+0000 7f6733fff640 1 -- 172.21.15.107:0/3984681548 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440cd3f0 con 0x560ef5bd4940 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.070+0000 7f6733fff640 1 -- 172.21.15.107:0/3984681548 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6744036030 con 0x560ef5bd4940 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.070+0000 7f6733fff640 1 -- 172.21.15.107:0/3984681548 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440ccf50 con 0x560ef5bd4940 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- 172.21.15.107:0/3984681548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 msgr2=0x560ef5be1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67440c9f60 tx=0x7f67440c8c30 comp rx=0 tx=0).stop 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- 172.21.15.107:0/3984681548 shutdown_connections 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5be1a00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc89b0 0x560ef5bd2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 --2- 172.21.15.107:0/3984681548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc8090 0x560ef5bd1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- 172.21.15.107:0/3984681548 >> 172.21.15.107:0/3984681548 conn(0x560ef5bc5ec0 msgr2=0x560ef5bccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- 172.21.15.107:0/3984681548 shutdown_connections 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- 172.21.15.107:0/3984681548 wait complete. 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 -- start start 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.071+0000 7f6753159740 1 Processor -- start 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc8090 0x560ef5bbe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bbe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f672c00cb20 con 0x560ef5bd4940 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f672c00df90 con 0x560ef5bc89b0 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f672c00d1f0 con 0x560ef5bc8090 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6753159740 10 client.0 _ll_register_callbacks cb 0x7f6740009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:43.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36372/0 (socket says 172.21.15.107:36372) 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 -- 172.21.15.107:0/1405158233 learned_addr learned my addr 172.21.15.107:0/1405158233 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f6749d9b640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bbe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674a59c640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc8090 0x560ef5bbe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 -- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc8090 msgr2=0x560ef5bbe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc8090 0x560ef5bbe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 -- 172.21.15.107:0/1405158233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 msgr2=0x560ef5bbe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bbe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.072+0000 7f674ad9d640 1 -- 172.21.15.107:0/1405158233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef5c66ef0 con 0x560ef5bd4940 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f674ad9d640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6744031920 tx=0x7f67440c9230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e560eb2ce65f7dc8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440cd3f0 con 0x560ef5bd4940 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f67440cd160 con 0x560ef5bd4940 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6744036030 con 0x560ef5bd4940 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67440d10b0 con 0x560ef5bd4940 2024-08-20T06:37:43.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 10 client.15498 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560ef5c68250 con 0x560ef5bd4940 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6744036030 con 0x560ef5bd4940 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f6744102ce0 con 0x560ef5bd4940 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 10 client.15498 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.073+0000 7f6753159740 20 client.15498 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5c68980 0x560ef5c6ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6753159740 4 client.15498 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ef5b34320 con 0x560ef5c68980 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f674a59c640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5c68980 0x560ef5c6ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6731ffb640 10 client.15498 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f674a59c640 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5c68980 0x560ef5c6ada0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2bcd747acc38be1 server_cookie=336529159f4498bb in_seq=0 out_seq=0 2024-08-20T06:37:43.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6731ffb640 1 -- 172.21.15.107:0/1405158233 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ef5b34320 con 0x560ef5c68980 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6731ffb640 10 client.15498 handle_command_reply: tid=0 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.074+0000 7f6753159740 1 client.15498 shutdown 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 msgr2=0x560ef5bbed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6744031920 tx=0x7f67440c9230 comp rx=0 tx=0).stop 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 shutdown_connections 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x560ef5c68980 0x560ef5c6ada0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560ef5bd4940 0x560ef5bbed40 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560ef5bc89b0 0x560ef5bbe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 --2- 172.21.15.107:0/1405158233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560ef5bc8090 0x560ef5bbe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 >> 172.21.15.107:0/1405158233 conn(0x560ef5bc5ec0 msgr2=0x560ef5bd2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6731ffb640 0 client.15498 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 shutdown_connections 2024-08-20T06:37:43.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 1 -- 172.21.15.107:0/1405158233 wait complete. 2024-08-20T06:37:43.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.075+0000 7f6753159740 20 client.15498 trim_cache size 0 max 16384 2024-08-20T06:37:43.081 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:37:43.081 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 msgr2=0x7f67340b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 0x7f67340b6270 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6740009e70 tx=0x7f6740005f90 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 msgr2=0x7f674c1a09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6744031040 tx=0x7f6744003f80 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 shutdown_connections 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f674c107d20 0x7f674c1a09c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f67340b3db0 0x7f67340b6270 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f674c1073a0 0x7f674c19b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 --2- 172.21.15.107:0/3950569078 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f674c101990 0x7f674c19b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 >> 172.21.15.107:0/3950569078 conn(0x7f674c0fd400 msgr2=0x7f674c06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 shutdown_connections 2024-08-20T06:37:43.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:43.076+0000 7f6750d60640 1 -- 172.21.15.107:0/3950569078 wait complete. 2024-08-20T06:37:43.800 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:37:44.116 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:44.493 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:44 smithi107 ceph-mon[18495]: pgmap v354: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:37:44.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 -- 172.21.15.107:0/420822602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 msgr2=0x7f95a410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 --2- 172.21.15.107:0/420822602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a410e520 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f959c009ca0 tx=0x7f959c02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 -- 172.21.15.107:0/420822602 shutdown_connections 2024-08-20T06:37:44.585 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 --2- 172.21.15.107:0/420822602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a410e520 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 --2- 172.21.15.107:0/420822602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 0x7f95a41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 --2- 172.21.15.107:0/420822602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 0x7f95a4101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 -- 172.21.15.107:0/420822602 >> 172.21.15.107:0/420822602 conn(0x7f95a40fd400 msgr2=0x7f95a40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 -- 172.21.15.107:0/420822602 shutdown_connections 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.578+0000 7f95a9f81640 1 -- 172.21.15.107:0/420822602 wait complete. 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 Processor -- start 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 -- start start 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 0x7f95a4198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 0x7f95a4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.586 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111920 con 0x7f95a4107d20 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41117a0 con 0x7f95a4101990 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.579+0000 7f95a9f81640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4111aa0 con 0x7f95a41073a0 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36394/0 (socket says 172.21.15.107:36394) 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 -- 172.21.15.107:0/3230559023 learned_addr learned my addr 172.21.15.107:0/3230559023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a37fe640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 0x7f95a4198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 0x7f95a4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 -- 172.21.15.107:0/3230559023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 msgr2=0x7f95a4199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 0x7f95a4199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 -- 172.21.15.107:0/3230559023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 msgr2=0x7f95a4198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 0x7f95a4198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 -- 172.21.15.107:0/3230559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a419efd0 con 0x7f95a4107d20 2024-08-20T06:37:44.587 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.580+0000 7f95a3fff640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f959c005cc0 tx=0x7f959c032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.581+0000 7f95a0ff9640 1 -- 172.21.15.107:0/3230559023 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c004040 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.581+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a419f260 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.581+0000 7f95a0ff9640 1 -- 172.21.15.107:0/3230559023 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f959c004670 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.581+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a4108570 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.581+0000 7f95a0ff9640 1 -- 172.21.15.107:0/3230559023 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c030cc0 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.583+0000 7f95ac37a740 1 Processor -- start 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a0ff9640 1 -- 172.21.15.107:0/3230559023 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f959c04e440 con 0x7f95a4107d20 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 -- start start 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90a9ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 0x560f90aa6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f90a11930 con 0x560f90a9b1a0 2024-08-20T06:37:44.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f90a00e30 con 0x560f90a9a880 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95ac37a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f90a04e70 con 0x560f90aa7130 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90a9ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 0x560f90aa6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:58830/0 (socket says 172.21.15.107:58830) 2024-08-20T06:37:44.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.584+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2774514348 learned_addr learned my addr 172.21.15.107:0/2774514348 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:44.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a0ff9640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 0x7f958c0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2774514348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 msgr2=0x560f90a9ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90a9ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2774514348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 msgr2=0x560f90aa6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 0x560f90aa6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2774514348 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f90760650 con 0x560f90aa7130 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a3fff640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 0x7f958c0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a37fe640 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 0x560f90aa6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a0ff9640 1 -- 172.21.15.107:0/3230559023 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f959c105130 con 0x7f95a4107d20 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9598017790 tx=0x7f9598009110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3fd03425fd06c7a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.585+0000 7f9589ffb640 1 -- 172.21.15.107:0/2774514348 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959801c020 con 0x560f90aa7130 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.586+0000 7f9589ffb640 1 -- 172.21.15.107:0/2774514348 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9598015df0 con 0x560f90aa7130 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.586+0000 7f9589ffb640 1 -- 172.21.15.107:0/2774514348 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598018ce0 con 0x560f90aa7130 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.586+0000 7f95a3fff640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 0x7f958c0b6220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f959c0cc040 tx=0x7f959c039000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.586+0000 7f95ac37a740 1 -- 172.21.15.107:0/2774514348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 msgr2=0x560f90aa9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.586+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9598017790 tx=0x7f9598009110 comp rx=0 tx=0).stop 2024-08-20T06:37:44.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 -- 172.21.15.107:0/2774514348 shutdown_connections 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90aa9520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9b1a0 0x560f90aa6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2774514348 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90a9ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 -- 172.21.15.107:0/2774514348 >> 172.21.15.107:0/2774514348 conn(0x560f90a986b0 msgr2=0x560f90a99850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 -- 172.21.15.107:0/2774514348 shutdown_connections 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 -- 172.21.15.107:0/2774514348 wait complete. 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 -- start start 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.587+0000 7f95ac37a740 1 Processor -- start 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90aa4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90aa45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560f90863700 con 0x560f90aa7130 2024-08-20T06:37:44.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560f90987810 con 0x560f90a9a880 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560f90aa9de0 con 0x560f90a9b1a0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95ac37a740 10 client.0 _ll_register_callbacks cb 0x560f909fa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36422/0 (socket says 172.21.15.107:36422) 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 -- 172.21.15.107:0/3887890236 learned_addr learned my addr 172.21.15.107:0/3887890236 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90aa4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a37fe640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90aa45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 -- 172.21.15.107:0/3887890236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 msgr2=0x560f90aa45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90aa45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 -- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 msgr2=0x560f90aa4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90aa4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.588+0000 7f95a2ffd640 1 -- 172.21.15.107:0/3887890236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f90abb9e0 con 0x560f90aa7130 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9598001eb0 tx=0x7f95980188c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5134fccd96afd78 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f9582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959801c030 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9598018b20 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f909b0150 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9598015760 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f95ac37a740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.589+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f90abe370 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560f909b0150 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9598021800 con 0x560f90aa7130 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 20 client.15510 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90abf6c0 0x560f90ac1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 4 client.15510 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:44.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f90a13880 con 0x560f90abf6c0 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.590+0000 7f95a37fe640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90abf6c0 0x560f90ac1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.591+0000 7f95a37fe640 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90abf6c0 0x560f90ac1ab0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a74d46fce96f87 server_cookie=9411133789c757f7 in_seq=0 out_seq=0 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.591+0000 7f9582ffd640 10 client.15510 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.591+0000 7f9582ffd640 1 -- 172.21.15.107:0/3887890236 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f90a13880 con 0x560f90abf6c0 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.591+0000 7f9582ffd640 10 client.15510 handle_command_reply: tid=0 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.591+0000 7f95ac37a740 1 client.15510 shutdown 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 msgr2=0x560f90aa4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9598001eb0 tx=0x7f95980188c0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 shutdown_connections 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90aa4ae0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90abf6c0 0x560f90ac1ab0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90aa45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f9582ffd640 0 client.15510 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 --2- 172.21.15.107:0/3887890236 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9a880 0x560f90aa4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 >> 172.21.15.107:0/3887890236 conn(0x560f90a986b0 msgr2=0x560f90a9e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 shutdown_connections 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 1 -- 172.21.15.107:0/3887890236 wait complete. 2024-08-20T06:37:44.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.592+0000 7f95ac37a740 20 client.15510 trim_cache size 0 max 16384 2024-08-20T06:37:44.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.605+0000 7f95ac37a740 1 Processor -- start 2024-08-20T06:37:44.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.605+0000 7f95ac37a740 1 -- start start 2024-08-20T06:37:44.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.605+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9b1a0 0x560f90aa5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90ac7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95ac37a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9590010d90 con 0x560f90a9a880 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95ac37a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f959000e660 con 0x560f90a9b1a0 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95ac37a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f959000c490 con 0x560f90aa7130 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36438/0 (socket says 172.21.15.107:36438) 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2881963048 learned_addr learned my addr 172.21.15.107:0/2881963048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a37fe640 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9b1a0 0x560f90aa5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2881963048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 msgr2=0x560f90ac7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90ac7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2881963048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9b1a0 msgr2=0x560f90aa5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9b1a0 0x560f90aa5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 -- 172.21.15.107:0/2881963048 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f90a99130 con 0x560f90a9a880 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.606+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95980093c0 tx=0x7f9598009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f5692be3c4b187 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95897fa640 1 -- 172.21.15.107:0/2881963048 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959801c040 con 0x560f90a9a880 2024-08-20T06:37:44.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95897fa640 1 -- 172.21.15.107:0/2881963048 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9598006ca0 con 0x560f90a9a880 2024-08-20T06:37:44.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95897fa640 1 -- 172.21.15.107:0/2881963048 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598009b40 con 0x560f90a9a880 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95ac37a740 1 -- 172.21.15.107:0/2881963048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 msgr2=0x560f90aa2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f95980093c0 tx=0x7f9598009d90 comp rx=0 tx=0).stop 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.607+0000 7f95ac37a740 1 -- 172.21.15.107:0/2881963048 shutdown_connections 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90aa7130 0x560f90ac7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90a9b1a0 0x560f90aa5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 --2- 172.21.15.107:0/2881963048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90a9a880 0x560f90aa2f50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 -- 172.21.15.107:0/2881963048 >> 172.21.15.107:0/2881963048 conn(0x560f90a986b0 msgr2=0x560f90aa5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 -- 172.21.15.107:0/2881963048 shutdown_connections 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 -- 172.21.15.107:0/2881963048 wait complete. 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 -- start start 2024-08-20T06:37:44.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.608+0000 7f95ac37a740 1 Processor -- start 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90b222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90abf6c0 0x560f90ab76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f959000cb20 con 0x560f90aa7130 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f959000df90 con 0x560f90abf6c0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f959000d1f0 con 0x560f90a9b1a0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95ac37a740 10 client.0 _ll_register_callbacks cb 0x7f95940093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36448/0 (socket says 172.21.15.107:36448) 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 -- 172.21.15.107:0/223639340 learned_addr learned my addr 172.21.15.107:0/223639340 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90abf6c0 0x560f90ab76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90b222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 -- 172.21.15.107:0/223639340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 msgr2=0x560f90b222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90b222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 -- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90abf6c0 msgr2=0x560f90ab76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90abf6c0 0x560f90ab76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 -- 172.21.15.107:0/223639340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f90ab7db0 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90b222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f95a37fe640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x560f90ac20e0 tx=0x7f9594009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4fdfd0d89f0522 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.609+0000 7f9582ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940155d0 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9594015770 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560f90a0f320 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9594015970 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 10 client.15516 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560f90b29cb0 con 0x560f90aa7130 2024-08-20T06:37:44.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9594026480 con 0x560f90aa7130 2024-08-20T06:37:44.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9594031400 con 0x560f90aa7130 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 10 client.15516 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 20 client.15516 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90b2a490 0x560f90b2c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 4 client.15516 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.610+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f90a0f320 con 0x560f90b2a490 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90b2a490 0x560f90b2c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f95a2ffd640 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90b2a490 0x560f90b2c8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef9bd7ac02717fb server_cookie=4a7337c8fadb4e11 in_seq=0 out_seq=0 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f9582ffd640 10 client.15516 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f9582ffd640 1 -- 172.21.15.107:0/223639340 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f90a0f320 con 0x560f90b2a490 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f9582ffd640 10 client.15516 handle_command_reply: tid=0 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f95ac37a740 1 client.15516 shutdown 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 msgr2=0x560f90b22830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.611+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x560f90ac20e0 tx=0x7f9594009110 comp rx=0 tx=0).stop 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 shutdown_connections 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560f90abf6c0 0x560f90ab76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560f90aa7130 0x560f90b22830 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560f90b2a490 0x560f90b2c8b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 --2- 172.21.15.107:0/223639340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560f90a9b1a0 0x560f90b222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f9582ffd640 0 client.15516 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 >> 172.21.15.107:0/223639340 conn(0x560f90a986b0 msgr2=0x560f90a9e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 shutdown_connections 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 1 -- 172.21.15.107:0/223639340 wait complete. 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.612+0000 7f95ac37a740 20 client.15516 trim_cache size 0 max 16384 2024-08-20T06:37:44.616 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 msgr2=0x7f958c0b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 0x7f958c0b6220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f959c0cc040 tx=0x7f959c039000 comp rx=0 tx=0).stop 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 msgr2=0x7f95a419e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f959c005cc0 tx=0x7f959c032c40 comp rx=0 tx=0).stop 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 shutdown_connections 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f95a4107d20 0x7f95a419e800 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f958c0b3d60 0x7f958c0b6220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95a41073a0 0x7f95a4199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 --2- 172.21.15.107:0/3230559023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f95a4101990 0x7f95a4198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:44.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 >> 172.21.15.107:0/3230559023 conn(0x7f95a40fd400 msgr2=0x7f95a4104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:44.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 shutdown_connections 2024-08-20T06:37:44.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:44.613+0000 7f95a9f81640 1 -- 172.21.15.107:0/3230559023 wait complete. 2024-08-20T06:37:44.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:44 smithi183 ceph-mon[22204]: pgmap v354: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:37:44.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:44 smithi193 ceph-mon[21944]: pgmap v354: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T06:37:46.318 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:37:46.318 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:37:46.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:46 smithi107 ceph-mon[18495]: pgmap v355: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:46.612 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:46.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:46 smithi183 ceph-mon[22204]: pgmap v355: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:46.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:46 smithi193 ceph-mon[21944]: pgmap v355: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/2079440072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41084b0 msgr2=0x7fe8d41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/2079440072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41084b0 0x7fe8d41088f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009900 tx=0x7fe8c402a410 comp rx=0 tx=0).stop 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/2079440072 shutdown_connections 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/2079440072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d4108e30 0x7fe8d410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/2079440072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d41084b0 0x7fe8d41088f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/2079440072 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.116+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/2079440072 >> 172.21.15.107:0/2079440072 conn(0x7fe8d40fe550 msgr2=0x7fe8d4100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.117+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/2079440072 shutdown_connections 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.117+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/2079440072 wait complete. 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.117+0000 7fe8dbe2f640 1 Processor -- start 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 -- start start 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d41084b0 0x7fe8d4199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41129a0 con 0x7fe8d4108e30 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4112820 con 0x7fe8d4102aa0 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8dbe2f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4112b20 con 0x7fe8d41084b0 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8da3cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8da3cc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55570/0 (socket says 172.21.15.107:55570) 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8da3cc640 1 -- 172.21.15.107:0/70372766 learned_addr learned my addr 172.21.15.107:0/70372766 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8da3cc640 1 -- 172.21.15.107:0/70372766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d41084b0 msgr2=0x7fe8d4199490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T06:37:47.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.118+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d41084b0 0x7fe8d4199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8da3cc640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d41084b0 0x7fe8d4199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8da3cc640 1 -- 172.21.15.107:0/70372766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 msgr2=0x7fe8d4198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8da3cc640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8da3cc640 1 -- 172.21.15.107:0/70372766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d419f000 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8da3cc640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cad0 tx=0x7fe8cc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8caffd640 1 -- 172.21.15.107:0/70372766 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00ae20 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8caffd640 1 -- 172.21.15.107:0/70372766 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8cc005020 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d419f2f0 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8caffd640 1 -- 172.21.15.107:0/70372766 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc00f6b0 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.119+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d4075130 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.122+0000 7fe8de228740 1 Processor -- start 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.123+0000 7fe8caffd640 1 -- 172.21.15.107:0/70372766 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7fe8cc0051d0 con 0x7fe8d4108e30 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.123+0000 7fe8de228740 1 -- start start 2024-08-20T06:37:47.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cd4c60 0x55b695cd7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8caffd640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 0x7fe8c00b6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:46200/0 (socket says 172.21.15.107:46200) 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8d93ca640 1 -- 172.21.15.107:0/3403428063 learned_addr learned my addr 172.21.15.107:0/3403428063 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b695c3e940 con 0x55b695cd4c60 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b695c2de40 con 0x55b695cc83b0 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b695c31e80 con 0x55b695cc8cd0 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8caffd640 1 -- 172.21.15.107:0/70372766 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe8cc0e4170 con 0x7fe8d4108e30 2024-08-20T06:37:47.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8da3cc640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 0x7fe8c00b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.124+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cd4c60 0x55b695cd7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 -- 172.21.15.107:0/3403428063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 msgr2=0x55b695cd4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 -- 172.21.15.107:0/3403428063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cd4c60 msgr2=0x55b695cd7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cd4c60 0x55b695cd7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 -- 172.21.15.107:0/3403428063 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b695bca520 con 0x55b695cc83b0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8da3cc640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 0x7fe8c00b6120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0aaef0 tx=0x7fe8cc00a890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005760 tx=0x7fe8c4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb8c4f2f03194eb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8bffff640 1 -- 172.21.15.107:0/3403428063 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4045070 con 0x55b695cc83b0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8bffff640 1 -- 172.21.15.107:0/3403428063 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c40044c0 con 0x55b695cc83b0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.125+0000 7fe8bffff640 1 -- 172.21.15.107:0/3403428063 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035040 con 0x55b695cc83b0 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 msgr2=0x55b695cc8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4005760 tx=0x7fe8c4000bc0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 shutdown_connections 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cd4c60 0x55b695cd7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 --2- 172.21.15.107:0/3403428063 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695cc8790 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 >> 172.21.15.107:0/3403428063 conn(0x55b695cc61e0 msgr2=0x55b695cc7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 shutdown_connections 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.126+0000 7fe8de228740 1 -- 172.21.15.107:0/3403428063 wait complete. 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.127+0000 7fe8de228740 1 -- start start 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.127+0000 7fe8de228740 1 Processor -- start 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cd4c60 0x55b695cd2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55578/0 (socket says 172.21.15.107:55578) 2024-08-20T06:37:47.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 -- 172.21.15.107:0/45199575 learned_addr learned my addr 172.21.15.107:0/45199575 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b695a90890 con 0x55b695cc83b0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b695bb4700 con 0x55b695cd4c60 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b695cd7950 con 0x55b695cc8cd0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8de228740 10 client.0 _ll_register_callbacks cb 0x55b695c296f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cd4c60 0x55b695cd2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 -- 172.21.15.107:0/45199575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 msgr2=0x55b695cd2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 -- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cd4c60 msgr2=0x55b695cd2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cd4c60 0x55b695cd2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d93ca640 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b69598d930 con 0x55b695cc83b0 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.128+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:47.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40319c0 tx=0x7fe8c4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25744fd558abff7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035030 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe8c4003f20 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b695c40890 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8c4045040 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8de228740 10 client.15528 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b695ce3580 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b695c40890 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.129+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8c403a040 con 0x55b695cc83b0 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:47.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 20 client.15528 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695ce48d0 0x55b695ce6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 4 client.15528 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b695c3c330 con 0x55b695ce48d0 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.130+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695ce48d0 0x55b695ce6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695ce48d0 0x55b695ce6cc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4f6b3332d27135 server_cookie=7f84ee2ae24ea47a in_seq=0 out_seq=0 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8bd7fa640 10 client.15528 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/45199575 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b695c3c330 con 0x55b695ce48d0 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8bd7fa640 10 client.15528 handle_command_reply: tid=0 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8de228740 1 client.15528 shutdown 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 msgr2=0x55b695cd1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.131+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40319c0 tx=0x7fe8c4009640 comp rx=0 tx=0).stop 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 shutdown_connections 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cd4c60 0x55b695cd2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695ce48d0 0x55b695ce6cc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cd2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 --2- 172.21.15.107:0/45199575 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695cd1cc0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 >> 172.21.15.107:0/45199575 conn(0x55b695cc61e0 msgr2=0x55b695ccbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8bd7fa640 0 client.15528 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 shutdown_connections 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 1 -- 172.21.15.107:0/45199575 wait complete. 2024-08-20T06:37:47.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.132+0000 7fe8de228740 20 client.15528 trim_cache size 0 max 16384 2024-08-20T06:37:47.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.161+0000 7fe8de228740 1 Processor -- start 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.161+0000 7fe8de228740 1 -- start start 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695ce9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41b2b80 con 0x55b695cc61e0 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41b0450 con 0x55b695cc83b0 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8de228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ae280 con 0x55b695cc8cd0 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55588/0 (socket says 172.21.15.107:55588) 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 -- 172.21.15.107:0/1926983554 learned_addr learned my addr 172.21.15.107:0/1926983554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:47.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695ce9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 -- 172.21.15.107:0/1926983554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 msgr2=0x55b695cceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 -- 172.21.15.107:0/1926983554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 msgr2=0x55b695ce9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695ce9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.162+0000 7fe8d93ca640 1 -- 172.21.15.107:0/1926983554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b695cc6c60 con 0x55b695cc61e0 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.163+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4033000 tx=0x7fe8c4009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebbeba4ebe8e40e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.163+0000 7fe8bf7fe640 1 -- 172.21.15.107:0/1926983554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c4035040 con 0x55b695cc61e0 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.163+0000 7fe8bf7fe640 1 -- 172.21.15.107:0/1926983554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c4042030 con 0x55b695cc61e0 2024-08-20T06:37:47.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.163+0000 7fe8bf7fe640 1 -- 172.21.15.107:0/1926983554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c404a680 con 0x55b695cc61e0 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- 172.21.15.107:0/1926983554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 msgr2=0x55b695ce9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4033000 tx=0x7fe8c4009800 comp rx=0 tx=0).stop 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- 172.21.15.107:0/1926983554 shutdown_connections 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695cceed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc83b0 0x55b695ce9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 --2- 172.21.15.107:0/1926983554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc61e0 0x55b695ce9900 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- 172.21.15.107:0/1926983554 >> 172.21.15.107:0/1926983554 conn(0x55b695cd3f20 msgr2=0x55b695ce0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- 172.21.15.107:0/1926983554 shutdown_connections 2024-08-20T06:37:47.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- 172.21.15.107:0/1926983554 wait complete. 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 -- start start 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.164+0000 7fe8de228740 1 Processor -- start 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc61e0 0x55b695cea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695ce7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41ae910 con 0x55b695cc83b0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41afd80 con 0x55b695cc61e0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d41aefe0 con 0x55b695cc8cd0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8de228740 10 client.0 _ll_register_callbacks cb 0x7fe8c4032d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55590/0 (socket says 172.21.15.107:55590) 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 -- 172.21.15.107:0/3974177010 learned_addr learned my addr 172.21.15.107:0/3974177010 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695ce7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc61e0 0x55b695cea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 -- 172.21.15.107:0/3974177010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 msgr2=0x55b695ce7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695ce7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 -- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc61e0 msgr2=0x55b695cea150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc61e0 0x55b695cea150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.165+0000 7fe8d9bcb640 1 -- 172.21.15.107:0/3974177010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b695d4e210 con 0x55b695cc83b0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8d9bcb640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55b695bb47c0 tx=0x7fe8d00157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40837f5ef0317596 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0015e50 con 0x55b695cc83b0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe8d0025a60 con 0x55b695cc83b0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c4042030 con 0x55b695cc83b0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe8d0025c60 con 0x55b695cc83b0 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8de228740 10 client.15534 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:47.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.166+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b695d4f4f0 con 0x55b695cc83b0 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe8c4042030 con 0x55b695cc83b0 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe8d00373b0 con 0x55b695cc83b0 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 10 client.15534 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 20 client.15534 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695cd4c60 0x55b695d51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 4 client.15534 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:37:47.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b695c39a70 con 0x55b695cd4c60 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.167+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695cd4c60 0x55b695d51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.168+0000 7fe8bd7fa640 10 client.15534 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.168+0000 7fe8d93ca640 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695cd4c60 0x55b695d51cc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7de042e42881b1 server_cookie=da95a56cc2ec0936 in_seq=0 out_seq=0 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.168+0000 7fe8bd7fa640 1 -- 172.21.15.107:0/3974177010 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b695c39a70 con 0x55b695cd4c60 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.168+0000 7fe8bd7fa640 10 client.15534 handle_command_reply: tid=0 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.168+0000 7fe8de228740 1 client.15534 shutdown 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.169+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 msgr2=0x55b695ce7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.169+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55b695bb47c0 tx=0x7fe8d00157f0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 shutdown_connections 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b695cd4c60 0x55b695d51cc0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b695cc8cd0 0x55b695ce7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b695cc83b0 0x55b695ce7640 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.173 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:37:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:37:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "64079ea1-4ff1-4e93-b218-854a8833727b", 2024-08-20T06:37:47.173 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:37:47.173 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8bd7fa640 0 client.15534 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 --2- 172.21.15.107:0/3974177010 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b695cc61e0 0x55b695cea150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 >> 172.21.15.107:0/3974177010 conn(0x55b695cd3f20 msgr2=0x55b695cc4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 shutdown_connections 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 1 -- 172.21.15.107:0/3974177010 wait complete. 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.170+0000 7fe8de228740 20 client.15534 trim_cache size 0 max 16384 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 msgr2=0x7fe8c00b6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 0x7fe8c00b6120 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc0aaef0 tx=0x7fe8cc00a890 comp rx=0 tx=0).stop 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 msgr2=0x7fe8d419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00cad0 tx=0x7fe8cc00cf90 comp rx=0 tx=0).stop 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 shutdown_connections 2024-08-20T06:37:47.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8d4108e30 0x7fe8d419e830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe8c00b3c60 0x7fe8c00b6120 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d41084b0 0x7fe8d4199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 --2- 172.21.15.107:0/70372766 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe8d4102aa0 0x7fe8d4198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 >> 172.21.15.107:0/70372766 conn(0x7fe8d40fe550 msgr2=0x7fe8d4100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 shutdown_connections 2024-08-20T06:37:47.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:47.171+0000 7fe8dbe2f640 1 -- 172.21.15.107:0/70372766 wait complete. 2024-08-20T06:37:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:47 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:37:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:47 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:47 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:37:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:47 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:47 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:37:47.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:47 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:47 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:37:47.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:47 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:47 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:37:47.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:47 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:47 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:37:47.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:47 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:37:47.873 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:37:48.217 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:37:48.609 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:48 smithi107 ceph-mon[18495]: pgmap v356: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:37:48.700 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 -- 172.21.15.107:0/1415226727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 msgr2=0x7f37ec101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 --2- 172.21.15.107:0/1415226727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec101d70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009900 tx=0x7f37e002a410 comp rx=0 tx=0).stop 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 -- 172.21.15.107:0/1415226727 shutdown_connections 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 --2- 172.21.15.107:0/1415226727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 0x7f37ec10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 --2- 172.21.15.107:0/1415226727 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 0x7f37ec1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 --2- 172.21.15.107:0/1415226727 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec101d70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 -- 172.21.15.107:0/1415226727 >> 172.21.15.107:0/1415226727 conn(0x7f37ec0fd400 msgr2=0x7f37ec0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 -- 172.21.15.107:0/1415226727 shutdown_connections 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.691+0000 7f37f2d05640 1 -- 172.21.15.107:0/1415226727 wait complete. 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.692+0000 7f37f2d05640 1 Processor -- start 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.692+0000 7f37f2d05640 1 -- start start 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 0x7f37ec19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.701 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 0x7f37ec1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec111850 con 0x7f37ec101990 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1116d0 con 0x7f37ec1073a0 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f2d05640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1119d0 con 0x7f37ec107d20 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55606/0 (socket says 172.21.15.107:55606) 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 -- 172.21.15.107:0/985808329 learned_addr learned my addr 172.21.15.107:0/985808329 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37ebfff640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 0x7f37ec19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f12a2640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 0x7f37ec1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 -- 172.21.15.107:0/985808329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 msgr2=0x7f37ec1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 0x7f37ec1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 -- 172.21.15.107:0/985808329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 msgr2=0x7f37ec19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 0x7f37ec19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.693+0000 7f37f0aa1640 1 -- 172.21.15.107:0/985808329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ec1a1140 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009230 tx=0x7f37e0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37e9ffb640 1 -- 172.21.15.107:0/985808329 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e003f030 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37e9ffb640 1 -- 172.21.15.107:0/985808329 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37e0003a80 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ec1a13d0 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37e9ffb640 1 -- 172.21.15.107:0/985808329 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0035070 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.694+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ec1a7af0 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37e9ffb640 1 -- 172.21.15.107:0/985808329 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) ==== 151111+0+0 (secure 0 0 0) 0x7f37e0042030 con 0x7f37ec101990 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 Processor -- start 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 -- start start 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e26005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26006150 0x562e26011ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562e25f7c8e0 con 0x562e260120e0 2024-08-20T06:37:48.703 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e25f6bde0 con 0x562e26005830 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f50fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e25f6fe20 con 0x562e26006150 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55622/0 (socket says 172.21.15.107:55622) 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37ebfff640 1 -- 172.21.15.107:0/1535662809 learned_addr learned my addr 172.21.15.107:0/1535662809 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.698+0000 7f37f12a2640 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e26005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26006150 0x562e26011ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37e9ffb640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 0x7f37dc0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 -- 172.21.15.107:0/1535662809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26006150 msgr2=0x562e26011ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26006150 0x562e26011ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 -- 172.21.15.107:0/1535662809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 msgr2=0x562e26005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e26005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 -- 172.21.15.107:0/1535662809 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e25ccb650 con 0x562e260120e0 2024-08-20T06:37:48.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37ebfff640 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f37d800bd10 tx=0x7f37d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98bec191c45c554 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37e9ffb640 1 -- 172.21.15.107:0/985808329 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37e0103f90 con 0x7f37ec101990 2024-08-20T06:37:48.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37f12a2640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 0x7f37dc0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37d6ffd640 1 -- 172.21.15.107:0/1535662809 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8015070 con 0x562e260120e0 2024-08-20T06:37:48.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37d6ffd640 1 -- 172.21.15.107:0/1535662809 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37d80118d0 con 0x562e260120e0 2024-08-20T06:37:48.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.699+0000 7f37d6ffd640 1 -- 172.21.15.107:0/1535662809 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8010890 con 0x562e260120e0 2024-08-20T06:37:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- 172.21.15.107:0/1535662809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 msgr2=0x562e260144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f12a2640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 0x7f37dc0b61a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37e4007fc0 tx=0x7f37e4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f37d800bd10 tx=0x7f37d800f730 comp rx=0 tx=0).stop 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- 172.21.15.107:0/1535662809 shutdown_connections 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e260120e0 0x562e260144d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26006150 0x562e26011ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 --2- 172.21.15.107:0/1535662809 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e26005c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- 172.21.15.107:0/1535662809 >> 172.21.15.107:0/1535662809 conn(0x562e26003660 msgr2=0x562e26004800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- 172.21.15.107:0/1535662809 shutdown_connections 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- 172.21.15.107:0/1535662809 wait complete. 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 -- start start 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.701+0000 7f37f50fe740 1 Processor -- start 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26006150 0x562e2600f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.708 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562e25dce700 con 0x562e26006150 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562e25ef27c0 con 0x562e26005830 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.702+0000 7f37f50fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562e26014d90 con 0x562e2600fd90 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37ebfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26006150 0x562e2600f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:46286/0 (socket says 172.21.15.107:46286) 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 -- 172.21.15.107:0/4292605629 learned_addr learned my addr 172.21.15.107:0/4292605629 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f50fe740 10 client.0 _ll_register_callbacks cb 0x562e25f656b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f12a2640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 -- 172.21.15.107:0/4292605629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 msgr2=0x562e2601e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 -- 172.21.15.107:0/4292605629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26006150 msgr2=0x562e2600f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26006150 0x562e2600f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 -- 172.21.15.107:0/4292605629 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2601e7e0 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000c00 tx=0x7f37e0003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5e2c7dadfebccb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0049070 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37e00cafa0 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.703+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562e25f1b100 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37e00ca4b0 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 10 client.25049 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e26021170 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x562e25f1b100 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37e004c060 con 0x562e26005830 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 10 client.25049 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 10 client.25049 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 20 client.25049 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e26024bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 4 client.25049 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.704+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e25f7e830 con 0x562e26022760 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.705+0000 7f37ebfff640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e26024bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.705+0000 7f37cbfff640 10 client.25049 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.705+0000 7f37ebfff640 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e26024bd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b62ed8a3a26911d3 server_cookie=6088e86ff685e9fe in_seq=0 out_seq=0 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37cbfff640 1 -- 172.21.15.107:0/4292605629 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e25f7e830 con 0x562e26022760 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37cbfff640 10 client.25049 handle_command_reply: tid=0 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 client.25049 shutdown 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 msgr2=0x562e2600f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000c00 tx=0x7f37e0003e60 comp rx=0 tx=0).stop 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 shutdown_connections 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e26024bd0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37cbfff640 0 client.25049 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:48.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26006150 0x562e2600f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 --2- 172.21.15.107:0/4292605629 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26005830 0x562e2600f310 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 >> 172.21.15.107:0/4292605629 conn(0x562e26003660 msgr2=0x562e26009200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 shutdown_connections 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 1 -- 172.21.15.107:0/4292605629 wait complete. 2024-08-20T06:37:48.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.706+0000 7f37f50fe740 20 client.25049 trim_cache size 0 max 16384 2024-08-20T06:37:48.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:48 smithi183 ceph-mon[22204]: pgmap v356: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:37:48.740 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 Processor -- start 2024-08-20T06:37:48.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 -- start start 2024-08-20T06:37:48.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.741 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26004e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b2b20 con 0x562e26005830 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1b03f0 con 0x562e26006150 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f50fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1ae220 con 0x562e2600fd90 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f0aa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55640/0 (socket says 172.21.15.107:55640) 2024-08-20T06:37:48.742 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f0aa1640 1 -- 172.21.15.107:0/2292483929 learned_addr learned my addr 172.21.15.107:0/2292483929 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26004e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.740+0000 7f37f12a2640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 -- 172.21.15.107:0/2292483929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 msgr2=0x562e2601dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 -- 172.21.15.107:0/2292483929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 msgr2=0x562e26004e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26004e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 -- 172.21.15.107:0/2292483929 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e26001aa0 con 0x562e26005830 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26004e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f12a2640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37e003d040 tx=0x7f37e00ca750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e2fea5e9efa2ae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37d67fc640 1 -- 172.21.15.107:0/2292483929 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e0049070 con 0x562e26005830 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37d67fc640 1 -- 172.21.15.107:0/2292483929 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37e00ca850 con 0x562e26005830 2024-08-20T06:37:48.743 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.741+0000 7f37d67fc640 1 -- 172.21.15.107:0/2292483929 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e00cf140 con 0x562e26005830 2024-08-20T06:37:48.744 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 -- 172.21.15.107:0/2292483929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 msgr2=0x562e2600f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f37e003d040 tx=0x7f37e00ca750 comp rx=0 tx=0).stop 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 -- 172.21.15.107:0/2292483929 shutdown_connections 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e2600fd90 0x562e2601dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26004e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2292483929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e26005830 0x562e2600f810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 -- 172.21.15.107:0/2292483929 >> 172.21.15.107:0/2292483929 conn(0x562e26003660 msgr2=0x562e26004770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.742+0000 7f37f50fe740 1 -- 172.21.15.107:0/2292483929 shutdown_connections 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.743+0000 7f37f50fe740 1 -- 172.21.15.107:0/2292483929 wait complete. 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.743+0000 7f37f50fe740 1 -- start start 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.743+0000 7f37f50fe740 1 Processor -- start 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.743+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26005830 0x562e26015620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26015b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1ae8b0 con 0x562e2600fd90 2024-08-20T06:37:48.745 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1afd20 con 0x562e26006150 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37ec1aef80 con 0x562e26005830 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f50fe740 10 client.0 _ll_register_callbacks cb 0x7f37e0003ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f12a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f12a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55644/0 (socket says 172.21.15.107:55644) 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f12a2640 1 -- 172.21.15.107:0/2621017048 learned_addr learned my addr 172.21.15.107:0/2621017048 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26015b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26005830 0x562e26015620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 -- 172.21.15.107:0/2621017048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26005830 msgr2=0x562e26015620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26005830 0x562e26015620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 -- 172.21.15.107:0/2621017048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 msgr2=0x562e2608b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 -- 172.21.15.107:0/2621017048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e2608b7e0 con 0x562e26006150 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37f12a2640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37ebfff640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26015b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562e26006610 tx=0x7f37d80103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46e8e6d52d7307e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.744+0000 7f37cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8015040 con 0x562e26006150 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37d8004240 con 0x562e26006150 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37e00ca850 con 0x562e26006150 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37d8034050 con 0x562e26006150 2024-08-20T06:37:48.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37f50fe740 10 client.25055 fetch_fsmap learned FSMap version 25 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e2608cac0 con 0x562e26006150 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.745+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37e00ca850 con 0x562e26006150 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37d8004880 con 0x562e26006150 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 10 client.25055 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 10 client.25055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 20 client.25055 populate_metadata read hostname 'smithi107' 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e2608f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 4 client.25055 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562e25f7a2d0 con 0x562e26022760 2024-08-20T06:37:48.749 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.746+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e2608f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f0aa1640 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e2608f0a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66f22373aa564ee server_cookie=82a8d50ba9d4c964 in_seq=0 out_seq=0 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37cbfff640 10 client.25055 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37cbfff640 1 -- 172.21.15.107:0/2621017048 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562e25f7a2d0 con 0x562e26022760 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37cbfff640 10 client.25055 handle_command_reply: tid=0 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 client.25055 shutdown 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 msgr2=0x562e26015b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26015b60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x562e26006610 tx=0x7f37d80103d0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.750 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 shutdown_connections 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562e2600fd90 0x562e2608b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562e26006150 0x562e26015b60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562e26022760 0x562e2608f0a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 --2- 172.21.15.107:0/2621017048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e26005830 0x562e26015620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37cbfff640 0 client.25055 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 >> 172.21.15.107:0/2621017048 conn(0x562e26003660 msgr2=0x562e26001f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 shutdown_connections 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 1 -- 172.21.15.107:0/2621017048 wait complete. 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.747+0000 7f37f50fe740 20 client.25055 trim_cache size 0 max 16384 2024-08-20T06:37:48.751 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "64079ea1-4ff1-4e93-b218-854a8833727b": { 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "64079ea1-4ff1-4e93-b218-854a8833727b", 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:37:48.752 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.748+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 msgr2=0x7f37dc0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.748+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 0x7f37dc0b61a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f37e4007fc0 tx=0x7f37e4004a40 comp rx=0 tx=0).stop 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.748+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 msgr2=0x7f37ec19b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.748+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f37e0009230 tx=0x7f37e0004270 comp rx=0 tx=0).stop 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 shutdown_connections 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37dc0b3ce0 0x7f37dc0b61a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37ec107d20 0x7f37ec1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37ec1073a0 0x7f37ec19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 --2- 172.21.15.107:0/985808329 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37ec101990 0x7f37ec19b090 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:37:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 >> 172.21.15.107:0/985808329 conn(0x7f37ec0fd400 msgr2=0x7f37ec06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:37:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 shutdown_connections 2024-08-20T06:37:48.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:37:48.749+0000 7f37f2d05640 1 -- 172.21.15.107:0/985808329 wait complete. 2024-08-20T06:37:48.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:48 smithi193 ceph-mon[21944]: pgmap v356: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-08-20T06:37:49.452 INFO:tasks.cephfs.filesystem:scrub status for tag:64079ea1-4ff1-4e93-b218-854a8833727b - {'path': '/', 'tag': '64079ea1-4ff1-4e93-b218-854a8833727b', 'options': 'recursive,force'} 2024-08-20T06:37:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:50 smithi183 ceph-mon[22204]: pgmap v357: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:37:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:50 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:37:50.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:50 smithi193 ceph-mon[21944]: pgmap v357: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:37:50.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:50 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:50 smithi107 ceph-mon[18495]: pgmap v357: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:37:50.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:50 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:37:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:52 smithi183 ceph-mon[22204]: pgmap v358: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T06:37:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:52 smithi193 ceph-mon[21944]: pgmap v358: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T06:37:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:52 smithi107 ceph-mon[18495]: pgmap v358: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to 'ef4275410d8483698016fe484b2a3c664080b4c1'. 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-08-20T06:37:54.005 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:54.006 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at ef4275410d8 Merge branch 'mds-conf-sorted' of https://github.com/batrick/ceph into wip-hemanth-testing-2024-08-19-1357 2024-08-20T06:37:54.009 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-20T06:37:54.038 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-20T06:37:54.040 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T06:37:54.040 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-20T06:37:54.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:54 smithi183 ceph-mon[22204]: pgmap v359: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-20T06:37:54.765 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-20T06:37:54.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:54 smithi193 ceph-mon[21944]: pgmap v359: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-20T06:37:54.822 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-20T06:37:54.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:54 smithi107 ceph-mon[18495]: pgmap v359: 97 pgs: 97 active+clean; 661 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-08-20T06:37:54.863 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T06:37:54.865 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T06:37:54.865 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-20T06:37:54.903 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T06:37:54.907 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:37:54.907 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-20T06:37:54.964 INFO:tasks.workunit:Running workunits matching suites/blogbench.sh on client.0... 2024-08-20T06:37:54.965 INFO:tasks.workunit:Running workunit suites/blogbench.sh... 2024-08-20T06:37:54.966 DEBUG:teuthology.orchestra.run.smithi107:workunit test suites/blogbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/blogbench.sh 2024-08-20T06:37:55.030 INFO:tasks.workunit.client.0.smithi107.stdout:getting blogbench 2024-08-20T06:37:55.030 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'getting blogbench' 2024-08-20T06:37:55.030 INFO:tasks.workunit.client.0.smithi107.stderr:+ wget http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-20T06:37:55.060 INFO:tasks.workunit.client.0.smithi107.stderr:--2024-08-20 06:37:55-- http://download.ceph.com/qa/blogbench-1.0.tar.bz2 2024-08-20T06:37:55.061 INFO:tasks.workunit.client.0.smithi107.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-08-20T06:37:55.082 INFO:tasks.workunit.client.0.smithi107.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-08-20T06:37:55.102 INFO:tasks.workunit.client.0.smithi107.stderr:HTTP request sent, awaiting response... 200 OK 2024-08-20T06:37:55.103 INFO:tasks.workunit.client.0.smithi107.stderr:Length: 103355 (101K) [application/octet-stream] 2024-08-20T06:37:55.104 INFO:tasks.workunit.client.0.smithi107.stderr:Saving to: ‘blogbench-1.0.tar.bz2’ 2024-08-20T06:37:55.108 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:55.145 INFO:tasks.workunit.client.0.smithi107.stderr: 0K .......... .......... .......... .......... .......... 49% 1.25M 0s 2024-08-20T06:37:55.164 INFO:tasks.workunit.client.0.smithi107.stderr: 50K .......... .......... .......... .......... .......... 99% 2.45M 0s 2024-08-20T06:37:55.164 INFO:tasks.workunit.client.0.smithi107.stderr: 100K 100% 1.74T=0.06s 2024-08-20T06:37:55.164 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:55.165 INFO:tasks.workunit.client.0.smithi107.stderr:2024-08-20 06:37:55 (1.67 MB/s) - ‘blogbench-1.0.tar.bz2’ saved [103355/103355] 2024-08-20T06:37:55.165 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-08-20T06:37:55.165 INFO:tasks.workunit.client.0.smithi107.stderr:+ tar -xvf blogbench-1.0.tar.bz2 2024-08-20T06:37:55.202 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/ 2024-08-20T06:37:55.203 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/m4/ 2024-08-20T06:37:55.205 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/m4/acx_pthread.m4 2024-08-20T06:37:55.207 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/ 2024-08-20T06:37:55.209 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/blogbench.8 2024-08-20T06:37:55.211 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/Makefile.am 2024-08-20T06:37:55.213 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/man/Makefile.in 2024-08-20T06:37:55.214 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/ 2024-08-20T06:37:55.216 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/mysnprintf.c 2024-08-20T06:37:55.218 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/mysnprintf.h 2024-08-20T06:37:55.220 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench.c 2024-08-20T06:37:55.222 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench.h 2024-08-20T06:37:55.224 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/fakesnprintf.c 2024-08-20T06:37:55.225 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/fakesnprintf.h 2024-08-20T06:37:55.227 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/blogbench_p.h 2024-08-20T06:37:55.229 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/bsd-getopt_long.c 2024-08-20T06:37:55.232 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/bsd-getopt_long.h 2024-08-20T06:37:55.234 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/Makefile.am 2024-08-20T06:37:55.236 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/Makefile.in 2024-08-20T06:37:55.239 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/process.c 2024-08-20T06:37:55.241 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/globals.h 2024-08-20T06:37:55.243 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/commenter.c 2024-08-20T06:37:55.246 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/rewriter.c 2024-08-20T06:37:55.248 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/reader.c 2024-08-20T06:37:55.250 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/writer.c 2024-08-20T06:37:55.251 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/src/helpers.c 2024-08-20T06:37:55.253 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/NEWS 2024-08-20T06:37:55.255 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/compile 2024-08-20T06:37:55.257 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/depcomp 2024-08-20T06:37:55.259 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/aclocal.m4 2024-08-20T06:37:55.262 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/README 2024-08-20T06:37:55.263 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/configure 2024-08-20T06:37:55.272 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/configure.ac 2024-08-20T06:37:55.273 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.guess 2024-08-20T06:37:55.276 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/install-sh 2024-08-20T06:37:55.278 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.sub 2024-08-20T06:37:55.280 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/missing 2024-08-20T06:37:55.282 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/mkinstalldirs 2024-08-20T06:37:55.284 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/Makefile.am 2024-08-20T06:37:55.286 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/Makefile.in 2024-08-20T06:37:55.288 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/config.h.in 2024-08-20T06:37:55.290 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/AUTHORS 2024-08-20T06:37:55.292 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/INSTALL 2024-08-20T06:37:55.294 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/ChangeLog 2024-08-20T06:37:55.295 INFO:tasks.workunit.client.0.smithi107.stdout:blogbench-1.0/COPYING 2024-08-20T06:37:55.297 INFO:tasks.workunit.client.0.smithi107.stderr:+ cd blogbench-1.0/ 2024-08-20T06:37:55.299 INFO:tasks.workunit.client.0.smithi107.stdout:making blogbench 2024-08-20T06:37:55.299 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'making blogbench' 2024-08-20T06:37:55.299 INFO:tasks.workunit.client.0.smithi107.stderr:+ ./configure 2024-08-20T06:37:55.405 INFO:tasks.workunit.client.0.smithi107.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-20T06:37:56.416 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether build environment is sane... yes 2024-08-20T06:37:56.433 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gawk... gawk 2024-08-20T06:37:56.454 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether make sets $(MAKE)... yes 2024-08-20T06:37:56.482 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gcc... gcc 2024-08-20T06:37:56.558 INFO:tasks.workunit.client.0.smithi107.stdout:checking for C compiler default output file name... a.out 2024-08-20T06:37:56.561 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether the C compiler works... yes 2024-08-20T06:37:56.564 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether we are cross compiling... no 2024-08-20T06:37:56.613 INFO:tasks.workunit.client.0.smithi107.stdout:checking for suffix of executables... 2024-08-20T06:37:56.656 INFO:tasks.workunit.client.0.smithi107.stdout:checking for suffix of object files... o 2024-08-20T06:37:56.699 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether we are using the GNU C compiler... yes 2024-08-20T06:37:56.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:56 smithi183 ceph-mon[22204]: pgmap v360: 97 pgs: 97 active+clean; 782 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-08-20T06:37:56.741 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether gcc accepts -g... yes 2024-08-20T06:37:56.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:56 smithi193 ceph-mon[21944]: pgmap v360: 97 pgs: 97 active+clean; 782 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-08-20T06:37:56.817 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gcc option to accept ANSI C... none needed 2024-08-20T06:37:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:56 smithi107 ceph-mon[18495]: pgmap v360: 97 pgs: 97 active+clean; 782 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 10 KiB/s wr, 5 op/s 2024-08-20T06:37:56.879 INFO:tasks.workunit.client.0.smithi107.stdout:checking for style of include used by make... GNU 2024-08-20T06:37:57.009 INFO:tasks.workunit.client.0.smithi107.stdout:checking dependency style of gcc... gcc3 2024-08-20T06:37:57.009 INFO:tasks.workunit.client.0.smithi107.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-08-20T06:37:57.083 INFO:tasks.workunit.client.0.smithi107.stdout:checking how to run the C preprocessor... gcc -E 2024-08-20T06:37:57.145 INFO:tasks.workunit.client.0.smithi107.stdout:checking for egrep... grep -E 2024-08-20T06:37:57.158 INFO:tasks.workunit.client.0.smithi107.stdout:checking for AIX... no 2024-08-20T06:37:57.259 INFO:tasks.workunit.client.0.smithi107.stdout:checking for library containing strerror... none required 2024-08-20T06:37:57.485 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ANSI C header files... yes 2024-08-20T06:37:57.536 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-08-20T06:37:57.593 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/types.h... yes 2024-08-20T06:37:57.654 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/stat.h... yes 2024-08-20T06:37:57.719 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdlib.h... yes 2024-08-20T06:37:57.780 INFO:tasks.workunit.client.0.smithi107.stdout:checking for string.h... yes 2024-08-20T06:37:57.848 INFO:tasks.workunit.client.0.smithi107.stdout:checking for memory.h... yes 2024-08-20T06:37:57.912 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strings.h... yes 2024-08-20T06:37:57.976 INFO:tasks.workunit.client.0.smithi107.stdout:checking for inttypes.h... yes 2024-08-20T06:37:58.041 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdint.h... yes 2024-08-20T06:37:58.106 INFO:tasks.workunit.client.0.smithi107.stdout:checking for unistd.h... yes 2024-08-20T06:37:58.122 INFO:tasks.workunit.client.0.smithi107.stdout:checking for memory.h... (cached) yes 2024-08-20T06:37:58.132 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strings.h... (cached) yes 2024-08-20T06:37:58.142 INFO:tasks.workunit.client.0.smithi107.stdout:checking for unistd.h... (cached) yes 2024-08-20T06:37:58.209 INFO:tasks.workunit.client.0.smithi107.stdout:checking locale.h usability... yes 2024-08-20T06:37:58.234 INFO:tasks.workunit.client.0.smithi107.stdout:checking locale.h presence... yes 2024-08-20T06:37:58.235 INFO:tasks.workunit.client.0.smithi107.stdout:checking for locale.h... yes 2024-08-20T06:37:58.255 INFO:tasks.workunit.client.0.smithi107.stdout:checking for stdlib.h... (cached) yes 2024-08-20T06:37:58.265 INFO:tasks.workunit.client.0.smithi107.stdout:checking for string.h... (cached) yes 2024-08-20T06:37:58.342 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/socket.h usability... yes 2024-08-20T06:37:58.370 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/socket.h presence... yes 2024-08-20T06:37:58.371 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/socket.h... yes 2024-08-20T06:37:58.440 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/time.h usability... yes 2024-08-20T06:37:58.466 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/time.h presence... yes 2024-08-20T06:37:58.467 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/time.h... yes 2024-08-20T06:37:58.535 INFO:tasks.workunit.client.0.smithi107.stdout:checking fcntl.h usability... yes 2024-08-20T06:37:58.562 INFO:tasks.workunit.client.0.smithi107.stdout:checking fcntl.h presence... yes 2024-08-20T06:37:58.563 INFO:tasks.workunit.client.0.smithi107.stdout:checking for fcntl.h... yes 2024-08-20T06:37:58.636 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/fcntl.h usability... yes 2024-08-20T06:37:58.663 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/fcntl.h presence... yes 2024-08-20T06:37:58.664 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/fcntl.h... yes 2024-08-20T06:37:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:37:58 smithi183 ceph-mon[22204]: pgmap v361: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 80 KiB/s wr, 13 op/s 2024-08-20T06:37:58.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:37:58 smithi193 ceph-mon[21944]: pgmap v361: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 80 KiB/s wr, 13 op/s 2024-08-20T06:37:58.809 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/param.h usability... yes 2024-08-20T06:37:58.842 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/param.h presence... yes 2024-08-20T06:37:58.843 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/param.h... yes 2024-08-20T06:37:58.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:37:58 smithi107 ceph-mon[18495]: pgmap v361: 97 pgs: 97 active+clean; 1.5 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 80 KiB/s wr, 13 op/s 2024-08-20T06:37:58.902 INFO:tasks.workunit.client.0.smithi107.stdout:checking ioctl.h usability... no 2024-08-20T06:37:58.931 INFO:tasks.workunit.client.0.smithi107.stdout:checking ioctl.h presence... no 2024-08-20T06:37:58.932 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ioctl.h... no 2024-08-20T06:37:58.995 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/ioctl.h usability... yes 2024-08-20T06:37:59.026 INFO:tasks.workunit.client.0.smithi107.stdout:checking sys/ioctl.h presence... yes 2024-08-20T06:37:59.027 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sys/ioctl.h... yes 2024-08-20T06:37:59.090 INFO:tasks.workunit.client.0.smithi107.stdout:checking return type of signal handlers... void 2024-08-20T06:37:59.150 INFO:tasks.workunit.client.0.smithi107.stdout:checking for size_t... yes 2024-08-20T06:37:59.198 INFO:tasks.workunit.client.0.smithi107.stdout:checking for dev_t... yes 2024-08-20T06:37:59.255 INFO:tasks.workunit.client.0.smithi107.stdout:checking for short... yes 2024-08-20T06:37:59.351 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of short... 2 2024-08-20T06:37:59.408 INFO:tasks.workunit.client.0.smithi107.stdout:checking for int... yes 2024-08-20T06:37:59.495 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of int... 4 2024-08-20T06:37:59.558 INFO:tasks.workunit.client.0.smithi107.stdout:checking for long... yes 2024-08-20T06:37:59.647 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of long... 8 2024-08-20T06:37:59.704 INFO:tasks.workunit.client.0.smithi107.stdout:checking for long long... yes 2024-08-20T06:37:59.795 INFO:tasks.workunit.client.0.smithi107.stdout:checking size of long long... 8 2024-08-20T06:37:59.850 INFO:tasks.workunit.client.0.smithi107.stdout:checking for ssize_t... yes 2024-08-20T06:37:59.899 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sig_atomic_t... yes 2024-08-20T06:37:59.966 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether gcc needs -traditional... no 2024-08-20T06:38:00.016 INFO:tasks.workunit.client.0.smithi107.stdout:checking for an ANSI C-conforming const... yes 2024-08-20T06:38:00.053 INFO:tasks.workunit.client.0.smithi107.stdout:checking for inline... inline 2024-08-20T06:38:00.134 INFO:tasks.workunit.client.0.smithi107.stdout:checking for working alloca.h... yes 2024-08-20T06:38:00.202 INFO:tasks.workunit.client.0.smithi107.stdout:checking for alloca... yes 2024-08-20T06:38:00.282 INFO:tasks.workunit.client.0.smithi107.stdout:checking for working memcmp... yes 2024-08-20T06:38:00.358 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strftime... yes 2024-08-20T06:38:00.447 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether lstat dereferences a symlink specified with a trailing slash... yes 2024-08-20T06:38:00.529 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether stat accepts an empty string... no 2024-08-20T06:38:00.603 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vprintf... yes 2024-08-20T06:38:00.678 INFO:tasks.workunit.client.0.smithi107.stdout:checking for _doprnt... no 2024-08-20T06:38:00.706 INFO:tasks.workunit.client.0.smithi107.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-08-20T06:38:00.721 INFO:tasks.workunit.client.0.smithi107.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-08-20T06:38:00.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:00 smithi183 ceph-mon[22204]: pgmap v362: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 122 KiB/s wr, 19 op/s 2024-08-20T06:38:00.798 INFO:tasks.workunit.client.0.smithi107.stdout:checking for the pthreads library -lpthreads... no 2024-08-20T06:38:00.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:00 smithi193 ceph-mon[21944]: pgmap v362: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 122 KiB/s wr, 19 op/s 2024-08-20T06:38:00.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:00 smithi107 ceph-mon[18495]: pgmap v362: 97 pgs: 97 active+clean; 2.0 MiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 122 KiB/s wr, 19 op/s 2024-08-20T06:38:00.870 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether pthreads work without any flags... yes 2024-08-20T06:38:00.934 INFO:tasks.workunit.client.0.smithi107.stdout:checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE 2024-08-20T06:38:00.934 INFO:tasks.workunit.client.0.smithi107.stdout:checking if more special flags are required for pthreads... no 2024-08-20T06:38:00.935 INFO:tasks.workunit.client.0.smithi107.stdout:checking for cc_r... gcc 2024-08-20T06:38:01.024 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setrlimit... yes 2024-08-20T06:38:01.109 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setproctitle... no 2024-08-20T06:38:01.202 INFO:tasks.workunit.client.0.smithi107.stdout:checking for getopt_long... yes 2024-08-20T06:38:01.283 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strtoull... yes 2024-08-20T06:38:01.364 INFO:tasks.workunit.client.0.smithi107.stdout:checking for strtoq... yes 2024-08-20T06:38:01.452 INFO:tasks.workunit.client.0.smithi107.stdout:checking for snprintf... yes 2024-08-20T06:38:01.541 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vsnprintf... yes 2024-08-20T06:38:01.626 INFO:tasks.workunit.client.0.smithi107.stdout:checking for vfprintf... yes 2024-08-20T06:38:01.707 INFO:tasks.workunit.client.0.smithi107.stdout:checking for gethostname... yes 2024-08-20T06:38:01.790 INFO:tasks.workunit.client.0.smithi107.stdout:checking for setlocale... yes 2024-08-20T06:38:01.871 INFO:tasks.workunit.client.0.smithi107.stdout:checking for sync... yes 2024-08-20T06:38:01.953 INFO:tasks.workunit.client.0.smithi107.stdout:checking whether snprintf is C99 conformant... done 2024-08-20T06:38:01.969 INFO:tasks.workunit.client.0.smithi107.stdout:configure: creating ./config.status 2024-08-20T06:38:02.208 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating Makefile 2024-08-20T06:38:02.229 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating src/Makefile 2024-08-20T06:38:02.253 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating man/Makefile 2024-08-20T06:38:02.269 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: creating config.h 2024-08-20T06:38:02.335 INFO:tasks.workunit.client.0.smithi107.stdout:config.status: executing depfiles commands 2024-08-20T06:38:02.505 INFO:tasks.workunit.client.0.smithi107.stderr:+ make 2024-08-20T06:38:02.518 INFO:tasks.workunit.client.0.smithi107.stdout:make all-recursive 2024-08-20T06:38:02.526 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T06:38:02.527 INFO:tasks.workunit.client.0.smithi107.stdout:Making all in src 2024-08-20T06:38:02.550 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-20T06:38:02.550 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT blogbench.o -MD -MP -MF ".deps/blogbench.Tpo" -c -o blogbench.o blogbench.c; \ 2024-08-20T06:38:02.550 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/blogbench.Tpo" ".deps/blogbench.Po"; else rm -f ".deps/blogbench.Tpo"; exit 1; fi 2024-08-20T06:38:02.652 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT process.o -MD -MP -MF ".deps/process.Tpo" -c -o process.o process.c; \ 2024-08-20T06:38:02.652 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/process.Tpo" ".deps/process.Po"; else rm -f ".deps/process.Tpo"; exit 1; fi 2024-08-20T06:38:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:02 smithi183 ceph-mon[22204]: pgmap v363: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 145 KiB/s wr, 37 op/s 2024-08-20T06:38:02.750 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT writer.o -MD -MP -MF ".deps/writer.Tpo" -c -o writer.o writer.c; \ 2024-08-20T06:38:02.750 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/writer.Tpo" ".deps/writer.Po"; else rm -f ".deps/writer.Tpo"; exit 1; fi 2024-08-20T06:38:02.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:02 smithi193 ceph-mon[21944]: pgmap v363: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 145 KiB/s wr, 37 op/s 2024-08-20T06:38:02.828 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT rewriter.o -MD -MP -MF ".deps/rewriter.Tpo" -c -o rewriter.o rewriter.c; \ 2024-08-20T06:38:02.828 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/rewriter.Tpo" ".deps/rewriter.Po"; else rm -f ".deps/rewriter.Tpo"; exit 1; fi 2024-08-20T06:38:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:02 smithi107 ceph-mon[18495]: pgmap v363: 97 pgs: 97 active+clean; 2.1 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 145 KiB/s wr, 37 op/s 2024-08-20T06:38:02.907 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT reader.o -MD -MP -MF ".deps/reader.Tpo" -c -o reader.o reader.c; \ 2024-08-20T06:38:02.907 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/reader.Tpo" ".deps/reader.Po"; else rm -f ".deps/reader.Tpo"; exit 1; fi 2024-08-20T06:38:02.984 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT commenter.o -MD -MP -MF ".deps/commenter.Tpo" -c -o commenter.o commenter.c; \ 2024-08-20T06:38:03.016 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/commenter.Tpo" ".deps/commenter.Po"; else rm -f ".deps/commenter.Tpo"; exit 1; fi 2024-08-20T06:38:03.069 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT helpers.o -MD -MP -MF ".deps/helpers.Tpo" -c -o helpers.o helpers.c; \ 2024-08-20T06:38:03.069 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/helpers.Tpo" ".deps/helpers.Po"; else rm -f ".deps/helpers.Tpo"; exit 1; fi 2024-08-20T06:38:03.172 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT bsd-getopt_long.o -MD -MP -MF ".deps/bsd-getopt_long.Tpo" -c -o bsd-getopt_long.o bsd-getopt_long.c; \ 2024-08-20T06:38:03.172 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/bsd-getopt_long.Tpo" ".deps/bsd-getopt_long.Po"; else rm -f ".deps/bsd-getopt_long.Tpo"; exit 1; fi 2024-08-20T06:38:03.232 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT fakesnprintf.o -MD -MP -MF ".deps/fakesnprintf.Tpo" -c -o fakesnprintf.o fakesnprintf.c; \ 2024-08-20T06:38:03.232 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/fakesnprintf.Tpo" ".deps/fakesnprintf.Po"; else rm -f ".deps/fakesnprintf.Tpo"; exit 1; fi 2024-08-20T06:38:03.295 INFO:tasks.workunit.client.0.smithi107.stdout:if gcc -DHAVE_CONFIG_H -I. -I. -I.. -g -O2 -MT mysnprintf.o -MD -MP -MF ".deps/mysnprintf.Tpo" -c -o mysnprintf.o mysnprintf.c; \ 2024-08-20T06:38:03.296 INFO:tasks.workunit.client.0.smithi107.stdout:then mv -f ".deps/mysnprintf.Tpo" ".deps/mysnprintf.Po"; else rm -f ".deps/mysnprintf.Tpo"; exit 1; fi 2024-08-20T06:38:03.352 INFO:tasks.workunit.client.0.smithi107.stdout:gcc -g -O2 -o blogbench blogbench.o process.o writer.o rewriter.o reader.o commenter.o helpers.o bsd-getopt_long.o fakesnprintf.o mysnprintf.o 2024-08-20T06:38:03.409 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/src' 2024-08-20T06:38:03.410 INFO:tasks.workunit.client.0.smithi107.stdout:Making all in man 2024-08-20T06:38:03.422 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-20T06:38:03.422 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Nothing to be done for 'all'. 2024-08-20T06:38:03.422 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0/man' 2024-08-20T06:38:03.430 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T06:38:03.433 INFO:tasks.workunit.client.0.smithi107.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T06:38:03.433 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/blogbench-1.0' 2024-08-20T06:38:03.435 INFO:tasks.workunit.client.0.smithi107.stderr:+ cd src 2024-08-20T06:38:03.436 INFO:tasks.workunit.client.0.smithi107.stderr:+ mkdir blogtest_in 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stdout:running blogbench 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'running blogbench' 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stderr:+ ./blogbench -d blogtest_in 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stdout:Frequency = 10 secs 2024-08-20T06:38:03.439 INFO:tasks.workunit.client.0.smithi107.stdout:Scratch dir = [blogtest_in] 2024-08-20T06:38:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:04 smithi183 ceph-mon[22204]: pgmap v364: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 466 KiB/s wr, 100 op/s 2024-08-20T06:38:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:04 smithi183 ceph-mon[22204]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-20T06:38:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:04 smithi193 ceph-mon[21944]: pgmap v364: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 466 KiB/s wr, 100 op/s 2024-08-20T06:38:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:04 smithi193 ceph-mon[21944]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-20T06:38:04.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:04 smithi107 ceph-mon[18495]: pgmap v364: 97 pgs: 97 active+clean; 5.4 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 466 KiB/s wr, 100 op/s 2024-08-20T06:38:04.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:04 smithi107 ceph-mon[18495]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-08-20T06:38:05.881 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 3 writers... 2024-08-20T06:38:05.882 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 1 rewriters... 2024-08-20T06:38:05.882 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 5 commenters... 2024-08-20T06:38:05.882 INFO:tasks.workunit.client.0.smithi107.stdout:Spawning 100 readers... 2024-08-20T06:38:05.884 INFO:tasks.workunit.client.0.smithi107.stdout:Benchmarking for 30 iterations. 2024-08-20T06:38:05.884 INFO:tasks.workunit.client.0.smithi107.stdout:The test will run during 5 minutes. 2024-08-20T06:38:05.884 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-08-20T06:38:05.884 INFO:tasks.workunit.client.0.smithi107.stdout: Nb blogs R articles W articles R pictures W pictures R comments W comments 2024-08-20T06:38:06.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:06 smithi183 ceph-mon[22204]: pgmap v365: 97 pgs: 97 active+clean; 5.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 481 KiB/s wr, 111 op/s 2024-08-20T06:38:06.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:06 smithi193 ceph-mon[21944]: pgmap v365: 97 pgs: 97 active+clean; 5.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 481 KiB/s wr, 111 op/s 2024-08-20T06:38:06.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:06 smithi107 ceph-mon[18495]: pgmap v365: 97 pgs: 97 active+clean; 5.4 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 481 KiB/s wr, 111 op/s 2024-08-20T06:38:08.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:08 smithi183 ceph-mon[22204]: pgmap v366: 97 pgs: 97 active+clean; 7.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 647 KiB/s wr, 134 op/s 2024-08-20T06:38:08.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:08 smithi193 ceph-mon[21944]: pgmap v366: 97 pgs: 97 active+clean; 7.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 647 KiB/s wr, 134 op/s 2024-08-20T06:38:08.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:08 smithi107 ceph-mon[18495]: pgmap v366: 97 pgs: 97 active+clean; 7.1 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 647 KiB/s wr, 134 op/s 2024-08-20T06:38:09.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:09 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:38:09.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:09 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:38:09.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:09 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 865 KiB/s wr, 146 op/s 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:38:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 865 KiB/s wr, 146 op/s 2024-08-20T06:38:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:38:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:38:11.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: pgmap v367: 97 pgs: 97 active+clean; 10 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 865 KiB/s wr, 146 op/s 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:38:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:38:12.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:12 smithi183 ceph-mon[22204]: pgmap v368: 97 pgs: 97 active+clean; 22 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-08-20T06:38:13.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:12 smithi193 ceph-mon[21944]: pgmap v368: 97 pgs: 97 active+clean; 22 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-08-20T06:38:13.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:12 smithi107 ceph-mon[18495]: pgmap v368: 97 pgs: 97 active+clean; 22 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.8 MiB/s wr, 174 op/s 2024-08-20T06:38:14.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:14 smithi183 ceph-mon[22204]: pgmap v369: 97 pgs: 97 active+clean; 28 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-08-20T06:38:15.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:14 smithi193 ceph-mon[21944]: pgmap v369: 97 pgs: 97 active+clean; 28 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-08-20T06:38:15.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:14 smithi107 ceph-mon[18495]: pgmap v369: 97 pgs: 97 active+clean; 28 MiB data, 475 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.4 MiB/s wr, 212 op/s 2024-08-20T06:38:15.884 INFO:tasks.workunit.client.0.smithi107.stdout: 5 14049 332 6923 341 4336 695 2024-08-20T06:38:16.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:16 smithi183 ceph-mon[22204]: pgmap v370: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-08-20T06:38:17.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:16 smithi193 ceph-mon[21944]: pgmap v370: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-08-20T06:38:17.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:16 smithi107 ceph-mon[18495]: pgmap v370: 97 pgs: 97 active+clean; 30 MiB data, 487 MiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-08-20T06:38:18.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:18 smithi183 ceph-mon[22204]: pgmap v371: 97 pgs: 97 active+clean; 34 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2024-08-20T06:38:19.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:18 smithi193 ceph-mon[21944]: pgmap v371: 97 pgs: 97 active+clean; 34 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2024-08-20T06:38:19.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:18 smithi107 ceph-mon[18495]: pgmap v371: 97 pgs: 97 active+clean; 34 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 864 KiB/s rd, 2.7 MiB/s wr, 177 op/s 2024-08-20T06:38:19.453 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:38:19.821 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:20.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2463826471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b410cff0 msgr2=0x7f40b410f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2463826471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b410cff0 0x7f40b410f3e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ca0 tx=0x7f40ac02ecd0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2463826471 shutdown_connections 2024-08-20T06:38:20.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2463826471 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b410cff0 0x7f40b410f3e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2463826471 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410a6c0 0x7f40b410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2463826471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b41015b0 0x7f40b4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2463826471 >> 172.21.15.107:0/2463826471 conn(0x7f40b4076b50 msgr2=0x7f40b4076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2463826471 shutdown_connections 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.360+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2463826471 wait complete. 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 Processor -- start 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 -- start start 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b410a6c0 0x7f40b4197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 0x7f40b419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4110770 con 0x7f40b41015b0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41105f0 con 0x7f40b410cff0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40bbf0b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41108f0 con 0x7f40b410a6c0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40b9ca7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40b9ca7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51542/0 (socket says 172.21.15.107:51542) 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40b9ca7640 1 -- 172.21.15.107:0/2903633237 learned_addr learned my addr 172.21.15.107:0/2903633237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40b94a6640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b410a6c0 0x7f40b4197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.361+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 0x7f40b419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 -- 172.21.15.107:0/2903633237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b410a6c0 msgr2=0x7f40b4197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b410a6c0 0x7f40b4197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 -- 172.21.15.107:0/2903633237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 msgr2=0x7f40b419c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 0x7f40b419c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 -- 172.21.15.107:0/2903633237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b419cdb0 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 0x7f40b419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d770 tx=0x7f40b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40aaffd640 1 -- 172.21.15.107:0/2903633237 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0017070 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b419d0a0 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40aaffd640 1 -- 172.21.15.107:0/2903633237 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40b00054b0 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a3760 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.362+0000 7f40aaffd640 1 -- 172.21.15.107:0/2903633237 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0004c60 con 0x7f40b41015b0 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.365+0000 7f40be304740 1 Processor -- start 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 -- start start 2024-08-20T06:38:20.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 0x555e771aea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e771198e0 con 0x555e771a3000 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e77108de0 con 0x555e771aef90 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40be304740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e7710ce20 con 0x555e771a26e0 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:41058/0 (socket says 172.21.15.107:41058) 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 -- 172.21.15.107:0/1578236845 learned_addr learned my addr 172.21.15.107:0/1578236845 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:20.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40aaffd640 1 -- 172.21.15.107:0/2903633237 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f40b0009c30 con 0x7f40b41015b0 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 -- 172.21.15.107:0/1578236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 msgr2=0x555e771a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b94a6640 1 -- 172.21.15.107:0/1578236845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 msgr2=0x555e771aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.366+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 0x555e771aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40b94a6640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 0x555e771aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40b94a6640 1 -- 172.21.15.107:0/1578236845 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e76e68650 con 0x555e771aef90 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 0x555e771aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40b94a6640 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009c60 tx=0x7f40a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcc8a430be085e98 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40aaffd640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 0x7f40a00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f409ffff640 1 -- 172.21.15.107:0/1578236845 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4004230 con 0x555e771aef90 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f409ffff640 1 -- 172.21.15.107:0/1578236845 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40a40043d0 con 0x555e771aef90 2024-08-20T06:38:20.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.367+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 0x7f40a00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40aaffd640 1 -- 172.21.15.107:0/2903633237 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40b00e3420 con 0x7f40b41015b0 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f409ffff640 1 -- 172.21.15.107:0/1578236845 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4004c60 con 0x555e771aef90 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 -- 172.21.15.107:0/1578236845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 msgr2=0x555e771b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009c60 tx=0x7f40a4011810 comp rx=0 tx=0).stop 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 0x7f40a00b6090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f40ac02f1f0 tx=0x7f40ac031040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 -- 172.21.15.107:0/1578236845 shutdown_connections 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771aef90 0x555e771b1380 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a3000 0x555e771aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 --2- 172.21.15.107:0/1578236845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.368+0000 7f40be304740 1 -- 172.21.15.107:0/1578236845 >> 172.21.15.107:0/1578236845 conn(0x555e771a0510 msgr2=0x555e771a16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.369+0000 7f40be304740 1 -- 172.21.15.107:0/1578236845 shutdown_connections 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.369+0000 7f40be304740 1 -- 172.21.15.107:0/1578236845 wait complete. 2024-08-20T06:38:20.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.369+0000 7f40be304740 1 -- start start 2024-08-20T06:38:20.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.369+0000 7f40be304740 1 Processor -- start 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a26e0 0x555e771ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 0x555e771ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e76f6b700 con 0x555e771a26e0 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e7708f7c0 con 0x555e771a3000 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e771b1ce0 con 0x555e771aef90 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40be304740 10 client.0 _ll_register_callbacks cb 0x555e771026b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b9ca7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a26e0 0x555e771ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:41068/0 (socket says 172.21.15.107:41068) 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 -- 172.21.15.107:0/499645972 learned_addr learned my addr 172.21.15.107:0/499645972 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 0x555e771ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 -- 172.21.15.107:0/499645972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 msgr2=0x555e771ac580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 0x555e771ac580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 -- 172.21.15.107:0/499645972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a26e0 msgr2=0x555e771ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a26e0 0x555e771ae9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 -- 172.21.15.107:0/499645972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e771bb0f0 con 0x555e771a3000 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 0x555e771ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f40b94a6640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009c30 tx=0x7f40a4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc347994c1653250 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.370+0000 7f409d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4027020 con 0x555e771a3000 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40a4003c10 con 0x555e771a3000 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e770b8100 con 0x555e771a3000 2024-08-20T06:38:20.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40a401e040 con 0x555e771a3000 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f40be304740 10 client.25067 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e771bd980 con 0x555e771a3000 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555e770b8100 con 0x555e771a3000 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40a4018070 con 0x555e771a3000 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f40be304740 10 client.25067 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.371+0000 7f40be304740 10 client.25067 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40be304740 20 client.25067 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e771bed30 0x555e771c1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40be304740 4 client.25067 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e7711b830 con 0x555e771bed30 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e771bed30 0x555e771c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f409d7fa640 10 client.25067 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:20.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.372+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e771bed30 0x555e771c1120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44a8a1d3a38f4ea9 server_cookie=57023b4f4b0388d in_seq=0 out_seq=0 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f409d7fa640 1 -- 172.21.15.107:0/499645972 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e7711b830 con 0x555e771bed30 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f409d7fa640 10 client.25067 handle_command_reply: tid=0 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 client.25067 shutdown 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 msgr2=0x555e771abdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009c30 tx=0x7f40a4003820 comp rx=0 tx=0).stop 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 shutdown_connections 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e771bed30 0x555e771c1120 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771aef90 0x555e771ac580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771abdf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 --2- 172.21.15.107:0/499645972 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771a26e0 0x555e771ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 >> 172.21.15.107:0/499645972 conn(0x555e771a0510 msgr2=0x555e771a60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.373+0000 7f409d7fa640 0 client.25067 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.374+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 shutdown_connections 2024-08-20T06:38:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.374+0000 7f40be304740 1 -- 172.21.15.107:0/499645972 wait complete. 2024-08-20T06:38:20.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.374+0000 7f40be304740 20 client.25067 trim_cache size 0 max 16384 2024-08-20T06:38:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 Processor -- start 2024-08-20T06:38:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 -- start start 2024-08-20T06:38:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 0x555e771a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a3000 0x555e771a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4098010d90 con 0x555e771aef90 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409800e660 con 0x555e771a26e0 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40be304740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f409800c490 con 0x555e771a3000 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40ba4a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40b9ca7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 0x555e771a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40ba4a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51566/0 (socket says 172.21.15.107:51566) 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40ba4a8640 1 -- 172.21.15.107:0/215730475 learned_addr learned my addr 172.21.15.107:0/215730475 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.396+0000 7f40b94a6640 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a3000 0x555e771a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 -- 172.21.15.107:0/215730475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a3000 msgr2=0x555e771a1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a3000 0x555e771a1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 -- 172.21.15.107:0/215730475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 msgr2=0x555e771c6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771c6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 -- 172.21.15.107:0/215730475 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e771a0f90 con 0x555e771a26e0 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 0x555e771a70a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555e771a5860 tx=0x7f40b000df00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b898bb1903e6aac4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f409f7fe640 1 -- 172.21.15.107:0/215730475 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0014070 con 0x555e771a26e0 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f409f7fe640 1 -- 172.21.15.107:0/215730475 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40b00b33e0 con 0x555e771a26e0 2024-08-20T06:38:20.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.397+0000 7f409f7fe640 1 -- 172.21.15.107:0/215730475 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0020040 con 0x555e771a26e0 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 -- 172.21.15.107:0/215730475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 msgr2=0x555e771a70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 0x555e771a70a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x555e771a5860 tx=0x7f40b000df00 comp rx=0 tx=0).stop 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 -- 172.21.15.107:0/215730475 shutdown_connections 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771c6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a3000 0x555e771a1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 --2- 172.21.15.107:0/215730475 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a26e0 0x555e771a70a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 -- 172.21.15.107:0/215730475 >> 172.21.15.107:0/215730475 conn(0x555e771a0510 msgr2=0x555e771a13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.406 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 -- 172.21.15.107:0/215730475 shutdown_connections 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.398+0000 7f40be304740 1 -- 172.21.15.107:0/215730475 wait complete. 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.399+0000 7f40be304740 1 -- start start 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.399+0000 7f40be304740 1 Processor -- start 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.399+0000 7f40be304740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771b74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.399+0000 7f40be304740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771a89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40be304740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40be304740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f409800cb20 con 0x555e771aef90 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40be304740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f409800df90 con 0x555e771a3000 2024-08-20T06:38:20.407 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40be304740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f409800d1f0 con 0x555e771a26e0 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40ba4a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771a89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40ba4a8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51576/0 (socket says 172.21.15.107:51576) 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40ba4a8640 1 -- 172.21.15.107:0/2773767084 learned_addr learned my addr 172.21.15.107:0/2773767084 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771b74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40be304740 10 client.0 _ll_register_callbacks cb 0x7f40a4009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 -- 172.21.15.107:0/2773767084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 msgr2=0x555e771b74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771b74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 -- 172.21.15.107:0/2773767084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 msgr2=0x555e771a8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:20.408 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 -- 172.21.15.107:0/2773767084 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e771a9570 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40ba4a8640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f40b94a6640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771a89c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40a4030040 tx=0x7f40a4023ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abae5ac258da8104 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.400+0000 7f409d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4027040 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f40a40233e0 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b00b33e0 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f40a4005c50 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 10 client.25073 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e77229940 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f40b00b33e0 con 0x555e771a3000 2024-08-20T06:38:20.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f40a4044020 con 0x555e771a3000 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 10 client.25073 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 10 client.25073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 20 client.25073 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e77229fd0 0x555e7722c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 4 client.25073 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.401+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e771172d0 con 0x555e77229fd0 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e77229fd0 0x555e7722c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f409d7fa640 10 client.25073 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f40b9ca7640 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e77229fd0 0x555e7722c3f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9236963b32a9ff93 server_cookie=3b1c105b727c4ce1 in_seq=0 out_seq=0 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f409d7fa640 1 -- 172.21.15.107:0/2773767084 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555e771172d0 con 0x555e77229fd0 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f409d7fa640 10 client.25073 handle_command_reply: tid=0 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f40be304740 1 client.25073 shutdown 2024-08-20T06:38:20.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 msgr2=0x555e771a89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.402+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771a89c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40a4030040 tx=0x7f40a4023ef0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 shutdown_connections 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e771aef90 0x555e771a8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e771a3000 0x555e771a89c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e77229fd0 0x555e7722c3f0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f409d7fa640 0 client.25073 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 --2- 172.21.15.107:0/2773767084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e771a26e0 0x555e771b74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 >> 172.21.15.107:0/2773767084 conn(0x555e771a0510 msgr2=0x555e771ad420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 shutdown_connections 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 1 -- 172.21.15.107:0/2773767084 wait complete. 2024-08-20T06:38:20.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.403+0000 7f40be304740 20 client.25073 trim_cache size 0 max 16384 2024-08-20T06:38:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:38:20.412 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:38:20.412 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:38:20.412 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 msgr2=0x7f40a00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 0x7f40a00b6090 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f40ac02f1f0 tx=0x7f40ac031040 comp rx=0 tx=0).stop 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 msgr2=0x7f40b4196d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40b000d770 tx=0x7f40b000dc30 comp rx=0 tx=0).stop 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 shutdown_connections 2024-08-20T06:38:20.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f40b410cff0 0x7f40b419c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f40a00b3bd0 0x7f40a00b6090 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40b410a6c0 0x7f40b4197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 --2- 172.21.15.107:0/2903633237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f40b41015b0 0x7f40b4196d00 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 >> 172.21.15.107:0/2903633237 conn(0x7f40b4076b50 msgr2=0x7f40b4100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 shutdown_connections 2024-08-20T06:38:20.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:20.404+0000 7f40bbf0b640 1 -- 172.21.15.107:0/2903633237 wait complete. 2024-08-20T06:38:20.977 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:20 smithi107 ceph-mon[18495]: pgmap v372: 97 pgs: 97 active+clean; 36 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-08-20T06:38:20.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:20 smithi183 ceph-mon[22204]: pgmap v372: 97 pgs: 97 active+clean; 36 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-08-20T06:38:21.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:20 smithi193 ceph-mon[21944]: pgmap v372: 97 pgs: 97 active+clean; 36 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 2.7 MiB/s wr, 170 op/s 2024-08-20T06:38:21.214 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:38:21.215 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:38:21.577 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:22.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.115+0000 7fe504f28640 1 -- 172.21.15.107:0/1824599953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5001026a0 msgr2=0x7fe500102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:22.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 --2- 172.21.15.107:0/1824599953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5001026a0 0x7fe500102a80 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009900 tx=0x7fe4f002a410 comp rx=0 tx=0).stop 2024-08-20T06:38:22.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 -- 172.21.15.107:0/1824599953 shutdown_connections 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 --2- 172.21.15.107:0/1824599953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 --2- 172.21.15.107:0/1824599953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe500107490 0x7fe5001078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 --2- 172.21.15.107:0/1824599953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe5001026a0 0x7fe500102a80 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 -- 172.21.15.107:0/1824599953 >> 172.21.15.107:0/1824599953 conn(0x7fe500076bd0 msgr2=0x7fe500076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 -- 172.21.15.107:0/1824599953 shutdown_connections 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.116+0000 7fe504f28640 1 -- 172.21.15.107:0/1824599953 wait complete. 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.117+0000 7fe504f28640 1 Processor -- start 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.117+0000 7fe504f28640 1 -- start start 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe504f28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe504f28640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe500107490 0x7fe500199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:22.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe504f28640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50019e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe4fe59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe504f28640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112940 con 0x7fe500107490 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe4fe59c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:42114/0 (socket says 172.21.15.107:42114) 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.118+0000 7fe4fe59c640 1 -- 172.21.15.107:0/1456740410 learned_addr learned my addr 172.21.15.107:0/1456740410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fdd9b640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe500107490 0x7fe500199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001127c0 con 0x7fe500107e10 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fed9d640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50019e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112ac0 con 0x7fe5001026a0 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fe59c640 1 -- 172.21.15.107:0/1456740410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 msgr2=0x7fe50019e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fe59c640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50019e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fe59c640 1 -- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe500107490 msgr2=0x7fe500199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fe59c640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe500107490 0x7fe500199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fe59c640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50019f070 con 0x7fe5001026a0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.119+0000 7fe4fed9d640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.120+0000 7fe4fe59c640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009230 tx=0x7fe4f0004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.120+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f003f030 con 0x7fe5001026a0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.120+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4f0003a80 con 0x7fe5001026a0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.120+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50019f300 con 0x7fe5001026a0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.120+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0035070 con 0x7fe5001026a0 2024-08-20T06:38:22.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.121+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a5a20 con 0x7fe5001026a0 2024-08-20T06:38:22.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.122+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe500102a80 con 0x7fe5001026a0 2024-08-20T06:38:22.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.124+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fe4f0042030 con 0x7fe5001026a0 2024-08-20T06:38:22.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.124+0000 7fe4ef7fe640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 0x7fe4e80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:22.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.125+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe4f0104540 con 0x7fe5001026a0 2024-08-20T06:38:22.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.125+0000 7fe4fdd9b640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 0x7fe4e80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:22.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.125+0000 7fe4fdd9b640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 0x7fe4e80b6090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ffa10 tx=0x7fe4f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:22.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.127+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe4f00cb540 con 0x7fe5001026a0 2024-08-20T06:38:22.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.520+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe50002e340 con 0x7fe5001026a0 2024-08-20T06:38:22.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.521+0000 7fe4ef7fe640 1 -- 172.21.15.107:0/1456740410 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7fe4f00cfa90 con 0x7fe5001026a0 2024-08-20T06:38:22.522 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:38:22.522 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"btime":"2024-08-20T06:36:56:241712+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:36:56.009554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:38:22.523 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-08-20T06:38:22.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.524+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 msgr2=0x7fe4e80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:22.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.524+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 0x7fe4e80b6090 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe5000ffa10 tx=0x7fe4f400b390 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.524+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 msgr2=0x7fe500198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.524+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0009230 tx=0x7fe4f0004270 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 shutdown_connections 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe500107e10 0x7fe50019e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe500107490 0x7fe500199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe4e80b3bd0 0x7fe4e80b6090 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 --2- 172.21.15.107:0/1456740410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5001026a0 0x7fe500198fc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 >> 172.21.15.107:0/1456740410 conn(0x7fe500076bd0 msgr2=0x7fe500100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 shutdown_connections 2024-08-20T06:38:22.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:22.525+0000 7fe504f28640 1 -- 172.21.15.107:0/1456740410 wait complete. 2024-08-20T06:38:22.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:22 smithi107 ceph-mon[18495]: pgmap v373: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.7 MiB/s wr, 169 op/s 2024-08-20T06:38:22.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:22 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1456740410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:22.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:22 smithi183 ceph-mon[22204]: pgmap v373: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.7 MiB/s wr, 169 op/s 2024-08-20T06:38:22.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:22 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1456740410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:23.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:22 smithi193 ceph-mon[21944]: pgmap v373: 97 pgs: 97 active+clean; 39 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 2.7 MiB/s wr, 169 op/s 2024-08-20T06:38:23.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:22 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1456740410' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:23.392 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:38:23.753 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:24.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 -- 172.21.15.107:0/1917180924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b410f0b0 msgr2=0x7f97b41114a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 --2- 172.21.15.107:0/1917180924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b410f0b0 0x7f97b41114a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac02a410 comp rx=0 tx=0).stop 2024-08-20T06:38:24.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 -- 172.21.15.107:0/1917180924 shutdown_connections 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 --2- 172.21.15.107:0/1917180924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b410f0b0 0x7f97b41114a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 --2- 172.21.15.107:0/1917180924 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b4075c70 0x7f97b410eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 --2- 172.21.15.107:0/1917180924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 0x7f97b4075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 -- 172.21.15.107:0/1917180924 >> 172.21.15.107:0/1917180924 conn(0x7f97b40fe540 msgr2=0x7f97b4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 -- 172.21.15.107:0/1917180924 shutdown_connections 2024-08-20T06:38:24.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.295+0000 7f97b907a640 1 -- 172.21.15.107:0/1917180924 wait complete. 2024-08-20T06:38:24.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.296+0000 7f97b907a640 1 Processor -- start 2024-08-20T06:38:24.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.296+0000 7f97b907a640 1 -- start start 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.296+0000 7f97b907a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 0x7f97b419b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b907a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b907a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b410f0b0 0x7f97b41a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b907a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4114b10 con 0x7f97b4075c70 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b907a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4114990 con 0x7f97b410f0b0 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b907a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97b4114c90 con 0x7f97b4075270 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51600/0 (socket says 172.21.15.107:51600) 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 -- 172.21.15.107:0/3799799011 learned_addr learned my addr 172.21.15.107:0/3799799011 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b359e640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b410f0b0 0x7f97b41a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 0x7f97b419b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 -- 172.21.15.107:0/3799799011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 msgr2=0x7f97b419b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 0x7f97b419b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 -- 172.21.15.107:0/3799799011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b410f0b0 msgr2=0x7f97b41a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b410f0b0 0x7f97b41a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 -- 172.21.15.107:0/3799799011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97b41a1130 con 0x7f97b4075c70 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.297+0000 7f97b259c640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f979c00d950 tx=0x7f979c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.298+0000 7f97abfff640 1 -- 172.21.15.107:0/3799799011 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c017030 con 0x7f97b4075c70 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.298+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97b41a1420 con 0x7f97b4075c70 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.298+0000 7f97abfff640 1 -- 172.21.15.107:0/3799799011 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f979c004020 con 0x7f97b4075c70 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.298+0000 7f97abfff640 1 -- 172.21.15.107:0/3799799011 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979c0055d0 con 0x7f97b4075c70 2024-08-20T06:38:24.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.298+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97b41a7cb0 con 0x7f97b4075c70 2024-08-20T06:38:24.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 Processor -- start 2024-08-20T06:38:24.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 -- start start 2024-08-20T06:38:24.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1db3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c143930 con 0x55bc6c1cc730 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c132e30 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.301+0000 7f97bb473740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c136e70 con 0x55bc6c1cd050 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97abfff640 1 -- 172.21.15.107:0/3799799011 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f979c009a30 con 0x7f97b4075c70 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51608/0 (socket says 172.21.15.107:51608) 2024-08-20T06:38:24.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b359e640 1 -- 172.21.15.107:0/2176822606 learned_addr learned my addr 172.21.15.107:0/2176822606 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:24.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b259c640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1db3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2176822606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1d8fe0 msgr2=0x55bc6c1db3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1db3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2176822606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 msgr2=0x55bc6c1ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2176822606 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6be92650 con 0x55bc6c1cd050 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.302+0000 7f97b359e640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d8aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a4010370 tx=0x7f97a4010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3872053d29a4b97a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97a97fa640 1 -- 172.21.15.107:0/2176822606 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4016070 con 0x55bc6c1cd050 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97a97fa640 1 -- 172.21.15.107:0/2176822606 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97a4004d20 con 0x55bc6c1cd050 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97a97fa640 1 -- 172.21.15.107:0/2176822606 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4005280 con 0x55bc6c1cd050 2024-08-20T06:38:24.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97abfff640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 0x7f97a00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97b359e640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 0x7f97a00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.303+0000 7f97abfff640 1 -- 172.21.15.107:0/3799799011 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f979c0e35d0 con 0x7f97b4075c70 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- 172.21.15.107:0/2176822606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 msgr2=0x55bc6c1d8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d8aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f97a4010370 tx=0x7f97a4010830 comp rx=0 tx=0).stop 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97b359e640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 0x7f97a00b6110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0096c0 tx=0x7f97ac031cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- 172.21.15.107:0/2176822606 shutdown_connections 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1db3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d8aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 --2- 172.21.15.107:0/2176822606 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cc730 0x55bc6c1ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- 172.21.15.107:0/2176822606 >> 172.21.15.107:0/2176822606 conn(0x55bc6c1ca560 msgr2=0x55bc6c1cb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- 172.21.15.107:0/2176822606 shutdown_connections 2024-08-20T06:38:24.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- 172.21.15.107:0/2176822606 wait complete. 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.304+0000 7f97bb473740 1 -- start start 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 Processor -- start 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bc6bf95700 con 0x55bc6c1cd050 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c0b9810 con 0x55bc6c1cc730 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bc6c1e43e0 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97bb473740 10 client.0 _ll_register_callbacks cb 0x55bc6c12c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51616/0 (socket says 172.21.15.107:51616) 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b259c640 1 -- 172.21.15.107:0/2173272029 learned_addr learned my addr 172.21.15.107:0/2173272029 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:24.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b359e640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2173272029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1d8fe0 msgr2=0x55bc6c1d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1d6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2173272029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 msgr2=0x55bc6c1d6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.305+0000 7f97b2d9d640 1 -- 172.21.15.107:0/2173272029 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6c1ed900 con 0x55bc6c1cc730 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d6030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a4006010 tx=0x7f97a4004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b35e48c71881a3b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4028060 con 0x55bc6c1cc730 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f97a4003d80 con 0x55bc6c1cc730 2024-08-20T06:38:24.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bc6c0e2150 con 0x55bc6c1cc730 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97a4016040 con 0x55bc6c1cc730 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 10 client.25091 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc6c1f02b0 con 0x55bc6c1cc730 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bc6c0e2150 con 0x55bc6c1cc730 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97a402d070 con 0x55bc6c1cc730 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 10 client.25091 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 10 client.25091 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.306+0000 7f97bb473740 20 client.25091 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c1f1710 0x55bc6c1f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f97bb473740 4 client.25091 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6c145880 con 0x55bc6c1f1710 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f97b259c640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c1f1710 0x55bc6c1f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f97b259c640 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c1f1710 0x55bc6c1f3b00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9216782fba01af45 server_cookie=2ab2021e6e8ae769 in_seq=0 out_seq=0 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.307+0000 7f979a7fc640 10 client.25091 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f979a7fc640 1 -- 172.21.15.107:0/2173272029 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc6c145880 con 0x55bc6c1f1710 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f979a7fc640 10 client.25091 handle_command_reply: tid=0 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 client.25091 shutdown 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 msgr2=0x55bc6c1d6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d6030 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a4006010 tx=0x7f97a4004c70 comp rx=0 tx=0).stop 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 shutdown_connections 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1d6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1cd050 0x55bc6c1d6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c1f1710 0x55bc6c1f3b00 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 --2- 172.21.15.107:0/2173272029 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d6030 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 >> 172.21.15.107:0/2173272029 conn(0x55bc6c1ca560 msgr2=0x55bc6c1d0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f979a7fc640 0 client.25091 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:24.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 shutdown_connections 2024-08-20T06:38:24.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 1 -- 172.21.15.107:0/2173272029 wait complete. 2024-08-20T06:38:24.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.308+0000 7f97bb473740 20 client.25091 trim_cache size 0 max 16384 2024-08-20T06:38:24.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 Processor -- start 2024-08-20T06:38:24.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 -- start start 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41ae6f0 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41abfc0 con 0x55bc6c1cc730 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97bb473740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41a9df0 con 0x55bc6c1cd050 2024-08-20T06:38:24.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51632/0 (socket says 172.21.15.107:51632) 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97b359e640 1 -- 172.21.15.107:0/3008112085 learned_addr learned my addr 172.21.15.107:0/3008112085 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.326+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b259c640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 -- 172.21.15.107:0/3008112085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 msgr2=0x55bc6c1c93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1c93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 -- 172.21.15.107:0/3008112085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 msgr2=0x55bc6c1d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 -- 172.21.15.107:0/3008112085 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6c1d4530 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97b359e640 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f434d4101fcd4941 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97a8ff9640 1 -- 172.21.15.107:0/3008112085 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac035070 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97a8ff9640 1 -- 172.21.15.107:0/3008112085 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97ac004300 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.327+0000 7f97a8ff9640 1 -- 172.21.15.107:0/3008112085 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ac050750 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 -- 172.21.15.107:0/3008112085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 msgr2=0x55bc6c1f90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac03f000 comp rx=0 tx=0).stop 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 -- 172.21.15.107:0/3008112085 shutdown_connections 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c1f90a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cd050 0x55bc6c1c93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 --2- 172.21.15.107:0/3008112085 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cc730 0x55bc6c1d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 -- 172.21.15.107:0/3008112085 >> 172.21.15.107:0/3008112085 conn(0x55bc6c1ca560 msgr2=0x55bc6c1c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 -- 172.21.15.107:0/3008112085 shutdown_connections 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.328+0000 7f97bb473740 1 -- 172.21.15.107:0/3008112085 wait complete. 2024-08-20T06:38:24.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 -- start start 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 Processor -- start 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 0x55bc6c1e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41aa480 con 0x55bc6c1d8fe0 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41ab8f0 con 0x55bc6c1cd050 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f97b41aab50 con 0x55bc6c1cc730 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97bb473740 10 client.0 _ll_register_callbacks cb 0x7f97a4009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b2d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b2d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:42158/0 (socket says 172.21.15.107:42158) 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51648/0 (socket says 172.21.15.107:51648) 2024-08-20T06:38:24.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b2d9d640 1 -- 172.21.15.107:0/1302929160 learned_addr learned my addr 172.21.15.107:0/1302929160 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.329+0000 7f97b259c640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 0x55bc6c1e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 -- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 msgr2=0x55bc6c1e5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 0x55bc6c1e5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 -- 172.21.15.107:0/1302929160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 msgr2=0x55bc6c25a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 -- 172.21.15.107:0/1302929160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6c25a770 con 0x55bc6c1cc730 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b259c640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 0x55bc6c1e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b359e640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97b2d9d640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a4011010 tx=0x7f97a40045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c6f826cc7fadfec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f979a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97a4033070 con 0x55bc6c1cc730 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ac004300 con 0x55bc6c1cc730 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f97a4012510 con 0x55bc6c1cc730 2024-08-20T06:38:24.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.330+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97a4028040 con 0x55bc6c1cc730 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 10 client.25045 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bc6c25bad0 con 0x55bc6c1cc730 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f97ac004300 con 0x55bc6c1cc730 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f97a4040050 con 0x55bc6c1cc730 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 10 client.25045 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 10 client.25045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 20 client.25045 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c25c130 0x55bc6c25e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 4 client.25045 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bc6c141320 con 0x55bc6c25c130 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.331+0000 7f97b259c640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c25c130 0x55bc6c25e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97b259c640 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c25c130 0x55bc6c25e520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9901792ffb73005d server_cookie=3f183c7023efce0a in_seq=0 out_seq=0 2024-08-20T06:38:24.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f979a7fc640 10 client.25045 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f979a7fc640 1 -- 172.21.15.107:0/1302929160 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bc6c141320 con 0x55bc6c25c130 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f979a7fc640 10 client.25045 handle_command_reply: tid=0 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 client.25045 shutdown 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 msgr2=0x55bc6c1e4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97a4011010 tx=0x7f97a40045b0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 shutdown_connections 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bc6c1d8fe0 0x55bc6c25a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55bc6c25c130 0x55bc6c25e520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bc6c1cd050 0x55bc6c1e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 --2- 172.21.15.107:0/1302929160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bc6c1cc730 0x55bc6c1e4ee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f979a7fc640 0 client.25045 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 >> 172.21.15.107:0/1302929160 conn(0x55bc6c1ca560 msgr2=0x55bc6c1c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.332+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 shutdown_connections 2024-08-20T06:38:24.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97bb473740 1 -- 172.21.15.107:0/1302929160 wait complete. 2024-08-20T06:38:24.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97bb473740 20 client.25045 trim_cache size 0 max 16384 2024-08-20T06:38:24.339 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:24.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 msgr2=0x7f97a00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 0x7f97a00b6110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0096c0 tx=0x7f97ac031cf0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 msgr2=0x7f97b419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.333+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f979c00d950 tx=0x7f979c00de10 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 shutdown_connections 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f97b410f0b0 0x7f97b41a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97b4075c70 0x7f97b419b5c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f97a00b3c50 0x7f97a00b6110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 --2- 172.21.15.107:0/3799799011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97b4075270 0x7f97b419b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 >> 172.21.15.107:0/3799799011 conn(0x7f97b40fe540 msgr2=0x7f97b40ff1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 shutdown_connections 2024-08-20T06:38:24.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:24.334+0000 7f97b907a640 1 -- 172.21.15.107:0/3799799011 wait complete. 2024-08-20T06:38:24.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:24 smithi183 ceph-mon[22204]: pgmap v374: 97 pgs: 97 active+clean; 41 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T06:38:25.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:24 smithi193 ceph-mon[21944]: pgmap v374: 97 pgs: 97 active+clean; 41 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T06:38:25.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:24 smithi107 ceph-mon[18495]: pgmap v374: 97 pgs: 97 active+clean; 41 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T06:38:25.158 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:38:25.519 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:25.884 INFO:tasks.workunit.client.0.smithi107.stdout: 7 14158 162 10849 112 5951 220 2024-08-20T06:38:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/2399403183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b41015b0 msgr2=0x7ff2b4101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/2399403183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b41015b0 0x7ff2b4101990 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009900 tx=0x7ff2b002a480 comp rx=0 tx=0).stop 2024-08-20T06:38:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/2399403183 shutdown_connections 2024-08-20T06:38:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/2399403183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b410f160 0x7ff2b4111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/2399403183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b4101ed0 0x7ff2b410ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/2399403183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b41015b0 0x7ff2b4101990 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/2399403183 >> 172.21.15.107:0/2399403183 conn(0x7ff2b40fd420 msgr2=0x7ff2b40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/2399403183 shutdown_connections 2024-08-20T06:38:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.347+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/2399403183 wait complete. 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 Processor -- start 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 -- start start 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b41015b0 0x7ff2b41038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 0x7ff2b4103e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 0x7ff2b4104360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4114b20 con 0x7ff2b410f160 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41149a0 con 0x7ff2b4101ed0 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.348+0000 7ff2bc6ed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4114ca0 con 0x7ff2b41015b0 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2bac8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 0x7ff2b4104360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 0x7ff2b4103e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2bac8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 0x7ff2b4104360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47590/0 (socket says 172.21.15.107:47590) 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2bac8a640 1 -- 172.21.15.107:0/1787615729 learned_addr learned my addr 172.21.15.107:0/1787615729 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 -- 172.21.15.107:0/1787615729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b41015b0 msgr2=0x7ff2b41038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b41015b0 0x7ff2b41038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 -- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 msgr2=0x7ff2b4104360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 0x7ff2b4104360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 -- 172.21.15.107:0/1787615729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41ab720 con 0x7ff2b4101ed0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 0x7ff2b4103e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c970 tx=0x7ff2a400ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2ab7fe640 1 -- 172.21.15.107:0/1787615729 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4007bb0 con 0x7ff2b4101ed0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41aba10 con 0x7ff2b4101ed0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.349+0000 7ff2ab7fe640 1 -- 172.21.15.107:0/1787615729 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2a4007d50 con 0x7ff2b4101ed0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.350+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41abfa0 con 0x7ff2b4101ed0 2024-08-20T06:38:26.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.350+0000 7ff2ab7fe640 1 -- 172.21.15.107:0/1787615729 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40186f0 con 0x7ff2b4101ed0 2024-08-20T06:38:26.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 Processor -- start 2024-08-20T06:38:26.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2ab7fe640 1 -- 172.21.15.107:0/1787615729 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff2a4018890 con 0x7ff2b4101ed0 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 -- start start 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x56507641dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 0x56507642c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x565076394930 con 0x56507642a130 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x565076383e30 con 0x56507641d880 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.353+0000 7ff2beae6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x565076387e70 con 0x56507641e1a0 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2b9c88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 0x56507642c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.357 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:60098/0 (socket says 172.21.15.107:60098) 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 -- 172.21.15.107:0/1749788870 learned_addr learned my addr 172.21.15.107:0/1749788870 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x56507641dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ab7fe640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 0x7ff2a00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 -- 172.21.15.107:0/1749788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 msgr2=0x56507641dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x56507641dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 -- 172.21.15.107:0/1749788870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 msgr2=0x56507642c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 0x56507642c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.358 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ba489640 1 -- 172.21.15.107:0/1749788870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650760e3650 con 0x56507641e1a0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 0x56507642c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.354+0000 7ff2ab7fe640 1 -- 172.21.15.107:0/1787615729 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff2a40e3ac0 con 0x7ff2b4101ed0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.355+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 0x7ff2a00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.355+0000 7ff2ba489640 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0032b00 tx=0x7ff2b0032b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf02ada795c9e28 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.355+0000 7ff2a8ff9640 1 -- 172.21.15.107:0/1749788870 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b003e030 con 0x56507641e1a0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.355+0000 7ff2a8ff9640 1 -- 172.21.15.107:0/1749788870 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b0032d50 con 0x56507641e1a0 2024-08-20T06:38:26.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.355+0000 7ff2a8ff9640 1 -- 172.21.15.107:0/1749788870 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0035040 con 0x56507641e1a0 2024-08-20T06:38:26.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- 172.21.15.107:0/1749788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 msgr2=0x565076429bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0032b00 tx=0x7ff2b0032b30 comp rx=0 tx=0).stop 2024-08-20T06:38:26.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 0x7ff2a00b6110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012450 tx=0x7ff2ac004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- 172.21.15.107:0/1749788870 shutdown_connections 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507642a130 0x56507642c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507641e1a0 0x565076429bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 --2- 172.21.15.107:0/1749788870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x56507641dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- 172.21.15.107:0/1749788870 >> 172.21.15.107:0/1749788870 conn(0x56507641b6b0 msgr2=0x56507641c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- 172.21.15.107:0/1749788870 shutdown_connections 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- 172.21.15.107:0/1749788870 wait complete. 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.356+0000 7ff2beae6740 1 -- start start 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 Processor -- start 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 0x5650764275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.364 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48218/0 (socket says 172.21.15.107:48218) 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2b9c88640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 0x5650764275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2ba489640 1 -- 172.21.15.107:0/2703524783 learned_addr learned my addr 172.21.15.107:0/2703524783 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076427b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5650761e6700 con 0x56507641e1a0 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56507630a810 con 0x56507641d880 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.357+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56507642cd80 con 0x56507642a130 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076427b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 -- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 msgr2=0x565076427b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076427b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2beae6740 10 client.0 _ll_register_callbacks cb 0x56507637d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 -- 172.21.15.107:0/2703524783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 msgr2=0x5650764270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650764361c0 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565076333150 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2ba489640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076427b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 0x5650764275f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56507641d830 tx=0x7ff2a40aaff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cacaba9a5d5bb3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff29dffb640 10 client.15603 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff29dffb640 1 -- 172.21.15.107:0/2703524783 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40aa140 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff29dffb640 1 -- 172.21.15.107:0/2703524783 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff2a4018b80 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.358+0000 7ff29dffb640 1 -- 172.21.15.107:0/2703524783 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2a40a6160 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 10 client.15603 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=25}) -- 0x565076438ad0 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff29dffb640 1 -- 172.21.15.107:0/2703524783 <== mon.0 v2:172.21.15.107:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2a40a9c30 con 0x56507641e1a0 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 10 client.15603 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:26.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 10 client.15603 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 20 client.15603 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x565076439f20 0x56507643c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 4 client.15603 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565076396880 con 0x565076439f20 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.359+0000 7ff2ba489640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x565076439f20 0x56507643c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.360+0000 7ff29dffb640 10 client.15603 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.360+0000 7ff2ba489640 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x565076439f20 0x56507643c380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25e76244dc2c1f7a server_cookie=55b87993ad8b3f4f in_seq=0 out_seq=0 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.360+0000 7ff29dffb640 1 -- 172.21.15.107:0/2703524783 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565076396880 con 0x565076439f20 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.360+0000 7ff29dffb640 10 client.15603 handle_command_reply: tid=0 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 client.15603 shutdown 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 msgr2=0x5650764275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 0x5650764275f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x56507641d830 tx=0x7ff2a40aaff0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 shutdown_connections 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x565076439f20 0x56507643c380 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076427b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641e1a0 0x5650764275f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 --2- 172.21.15.107:0/2703524783 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 >> 172.21.15.107:0/2703524783 conn(0x56507641b6b0 msgr2=0x565076421250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff29dffb640 0 client.15603 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 shutdown_connections 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 1 -- 172.21.15.107:0/2703524783 wait complete. 2024-08-20T06:38:26.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.361+0000 7ff2beae6740 20 client.15603 trim_cache size 0 max 16384 2024-08-20T06:38:26.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 Processor -- start 2024-08-20T06:38:26.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 -- start start 2024-08-20T06:38:26.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076441c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565076442190 0x5650764270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff298010d80 con 0x565076442190 2024-08-20T06:38:26.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800e650 con 0x56507641d880 2024-08-20T06:38:26.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.379+0000 7ff2beae6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800c480 con 0x56507642a130 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48220/0 (socket says 172.21.15.107:48220) 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 -- 172.21.15.107:0/3122776925 learned_addr learned my addr 172.21.15.107:0/3122776925 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076441c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565076442190 0x5650764270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 -- 172.21.15.107:0/3122776925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 msgr2=0x565076441c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076441c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 -- 172.21.15.107:0/3122776925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565076442190 msgr2=0x5650764270b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565076442190 0x5650764270b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 -- 172.21.15.107:0/3122776925 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56507641b4e0 con 0x56507641d880 2024-08-20T06:38:26.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.380+0000 7ff2ba489640 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00057c0 tx=0x7ff2b003c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16619a6c6a3c8179 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.381+0000 7ff29ffff640 1 -- 172.21.15.107:0/3122776925 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b003e040 con 0x56507641d880 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.381+0000 7ff29ffff640 1 -- 172.21.15.107:0/3122776925 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2b0031c70 con 0x56507641d880 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.381+0000 7ff29ffff640 1 -- 172.21.15.107:0/3122776925 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0035040 con 0x56507641d880 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.381+0000 7ff2beae6740 1 -- 172.21.15.107:0/3122776925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 msgr2=0x5650764218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.381+0000 7ff2beae6740 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00057c0 tx=0x7ff2b003c040 comp rx=0 tx=0).stop 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 -- 172.21.15.107:0/3122776925 shutdown_connections 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x565076442190 0x5650764270b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507642a130 0x565076441c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 --2- 172.21.15.107:0/3122776925 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507641d880 0x5650764218f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 -- 172.21.15.107:0/3122776925 >> 172.21.15.107:0/3122776925 conn(0x56507641b6b0 msgr2=0x565076428f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 -- 172.21.15.107:0/3122776925 shutdown_connections 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 -- 172.21.15.107:0/3122776925 wait complete. 2024-08-20T06:38:26.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 -- start start 2024-08-20T06:38:26.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.382+0000 7ff2beae6740 1 Processor -- start 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 0x565076432580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 0x56507642da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 0x56507642df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 0x565076432580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800cb10 con 0x56507641d880 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2ba489640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 0x565076432580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47618/0 (socket says 172.21.15.107:47618) 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2ba489640 1 -- 172.21.15.107:0/100136023 learned_addr learned my addr 172.21.15.107:0/100136023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 0x56507642da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800df80 con 0x56507642a130 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff29800d1e0 con 0x565076442190 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 0x56507642df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.389 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 10 client.0 _ll_register_callbacks cb 0x7ff2a40e7130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 -- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 msgr2=0x56507642df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 0x56507642df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 -- 172.21.15.107:0/100136023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 msgr2=0x565076432580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 0x565076432580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2b9c88640 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56507642e600 con 0x56507642a130 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.383+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2b0031c70 con 0x56507642a130 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff2bac8a640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 0x56507642df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff2b9c88640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 0x56507642da50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40a7c40 tx=0x7ff2a4091000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c7f4571dd78b54 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff29dffb640 10 client.25115 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff29dffb640 1 -- 172.21.15.107:0/100136023 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a40aa140 con 0x56507642a130 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff29dffb640 1 -- 172.21.15.107:0/100136023 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff2a40a6340 con 0x56507642a130 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff29dffb640 1 -- 172.21.15.107:0/100136023 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff2a40a6540 con 0x56507642a130 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff2beae6740 10 client.25115 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:26.390 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5650764a49d0 con 0x56507642a130 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.384+0000 7ff29dffb640 1 -- 172.21.15.107:0/100136023 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff2a4090ac0 con 0x56507642a130 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 10 client.25115 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 10 client.25115 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 20 client.25115 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5650764a4e50 0x5650764a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2ba489640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5650764a4e50 0x5650764a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 4 client.25115 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565076392320 con 0x5650764a4e50 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff29dffb640 10 client.25115 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.385+0000 7ff2ba489640 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5650764a4e50 0x5650764a7240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e93603bd0d8a8cfc server_cookie=460f34d3886ece19 in_seq=0 out_seq=0 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff29dffb640 1 -- 172.21.15.107:0/100136023 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565076392320 con 0x5650764a4e50 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff29dffb640 10 client.25115 handle_command_reply: tid=0 2024-08-20T06:38:26.391 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 client.25115 shutdown 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 msgr2=0x56507642da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 0x56507642da50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40a7c40 tx=0x7ff2a4091000 comp rx=0 tx=0).stop 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 shutdown_connections 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5650764a4e50 0x5650764a7240 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x565076442190 0x56507642df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff29dffb640 0 client.25115 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56507642a130 0x56507642da50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 --2- 172.21.15.107:0/100136023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56507641d880 0x565076432580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 >> 172.21.15.107:0/100136023 conn(0x56507641b6b0 msgr2=0x5650764270b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 shutdown_connections 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 1 -- 172.21.15.107:0/100136023 wait complete. 2024-08-20T06:38:26.392 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.386+0000 7ff2beae6740 20 client.25115 trim_cache size 0 max 16384 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 msgr2=0x7ff2a00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 0x7ff2a00b6110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012450 tx=0x7ff2ac004a40 comp rx=0 tx=0).stop 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 msgr2=0x7ff2b4103e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 0x7ff2b4103e20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400c970 tx=0x7ff2a400ce30 comp rx=0 tx=0).stop 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 shutdown_connections 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2b410f160 0x7ff2b4104360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2b4101ed0 0x7ff2b4103e20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.393 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2a00b3c50 0x7ff2a00b6110 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 --2- 172.21.15.107:0/1787615729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2b41015b0 0x7ff2b41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:26.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.387+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 >> 172.21.15.107:0/1787615729 conn(0x7ff2b40fd420 msgr2=0x7ff2b40ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:26.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.388+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 shutdown_connections 2024-08-20T06:38:26.394 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:26.388+0000 7ff2bc6ed640 1 -- 172.21.15.107:0/1787615729 wait complete. 2024-08-20T06:38:26.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:26 smithi183 ceph-mon[22204]: pgmap v375: 97 pgs: 97 active+clean; 42 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-08-20T06:38:27.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:26 smithi193 ceph-mon[21944]: pgmap v375: 97 pgs: 97 active+clean; 42 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-08-20T06:38:27.097 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:26 smithi107 ceph-mon[18495]: pgmap v375: 97 pgs: 97 active+clean; 42 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-08-20T06:38:27.133 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:38:27.491 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 -- 172.21.15.107:0/4264937155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0107d20 msgr2=0x7f37b010e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 --2- 172.21.15.107:0/4264937155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0107d20 0x7f37b010e520 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009ca0 tx=0x7f37a802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 -- 172.21.15.107:0/4264937155 shutdown_connections 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 --2- 172.21.15.107:0/4264937155 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0107d20 0x7f37b010e520 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 --2- 172.21.15.107:0/4264937155 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b01073a0 0x7f37b01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 --2- 172.21.15.107:0/4264937155 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b0101990 0x7f37b0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 -- 172.21.15.107:0/4264937155 >> 172.21.15.107:0/4264937155 conn(0x7f37b00fd400 msgr2=0x7f37b00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 -- 172.21.15.107:0/4264937155 shutdown_connections 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.081+0000 7f37b4a43640 1 -- 172.21.15.107:0/4264937155 wait complete. 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 Processor -- start 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 -- start start 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 0x7f37b0074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b0107d20 0x7f37b006d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01129f0 con 0x7f37b0101990 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0112870 con 0x7f37b01073a0 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.082+0000 7f37b4a43640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37b0112b70 con 0x7f37b0107d20 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47634/0 (socket says 172.21.15.107:47634) 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37add9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 0x7f37b0074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 -- 172.21.15.107:0/215823811 learned_addr learned my addr 172.21.15.107:0/215823811 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37aed9d640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b0107d20 0x7f37b006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 -- 172.21.15.107:0/215823811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b0107d20 msgr2=0x7f37b006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b0107d20 0x7f37b006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 -- 172.21.15.107:0/215823811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 msgr2=0x7f37b0074a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 0x7f37b0074a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 -- 172.21.15.107:0/215823811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37b006e150 con 0x7f37b0101990 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37add9b640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 0x7f37b0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f37ae59c640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9f0 tx=0x7f37a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.083+0000 7f379f7fe640 1 -- 172.21.15.107:0/215823811 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a000ccc0 con 0x7f37b0101990 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.084+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37b006e440 con 0x7f37b0101990 2024-08-20T06:38:28.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.084+0000 7f379f7fe640 1 -- 172.21.15.107:0/215823811 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a00108c0 con 0x7f37b0101990 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.084+0000 7f379f7fe640 1 -- 172.21.15.107:0/215823811 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a0018660 con 0x7f37b0101990 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.084+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37b0073360 con 0x7f37b0101990 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.087+0000 7f37b6e3c740 1 Processor -- start 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.087+0000 7f379f7fe640 1 -- 172.21.15.107:0/215823811 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f37a0010430 con 0x7f37b0101990 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 -- start start 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c7c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d891be4930 con 0x55d891c6e1a0 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d891bd3e30 con 0x55d891c6d880 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d891bd7e70 con 0x55d891c7a130 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37ae59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48262/0 (socket says 172.21.15.107:48262) 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 -- 172.21.15.107:0/3746124568 learned_addr learned my addr 172.21.15.107:0/3746124568 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37add9b640 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c7c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 -- 172.21.15.107:0/3746124568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 msgr2=0x55d891c7c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c7c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 -- 172.21.15.107:0/3746124568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 msgr2=0x55d891c79bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c79bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.088+0000 7f37aed9d640 1 -- 172.21.15.107:0/3746124568 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d891933650 con 0x55d891c6d880 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f37ae59c640 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c79bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f379f7fe640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 0x7f37980b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009430 tx=0x7f37a80314e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2ba1eb0478bd7c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f379f7fe640 1 -- 172.21.15.107:0/215823811 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37a00e4080 con 0x7f37b0101990 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f37ae59c640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 0x7f37980b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.099 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f379cff9640 1 -- 172.21.15.107:0/3746124568 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8031980 con 0x55d891c6d880 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.089+0000 7f379cff9640 1 -- 172.21.15.107:0/3746124568 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a8036040 con 0x55d891c6d880 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.090+0000 7f379cff9640 1 -- 172.21.15.107:0/3746124568 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8004500 con 0x55d891c6d880 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.090+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3746124568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 msgr2=0x55d891c6dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.090+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009430 tx=0x7f37a80314e0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.090+0000 7f37ae59c640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 0x7f37980b6090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f37a00a7050 tx=0x7f37a0010ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3746124568 shutdown_connections 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c7c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c79bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3746124568 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c6dc60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3746124568 >> 172.21.15.107:0/3746124568 conn(0x55d891c6b6b0 msgr2=0x55d891c6c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3746124568 shutdown_connections 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.091+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3746124568 wait complete. 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.092+0000 7f37b6e3c740 1 -- start start 2024-08-20T06:38:28.100 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.092+0000 7f37b6e3c740 1 Processor -- start 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.092+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6d880 0x55d891c77150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.092+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.092+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891c77bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d891a36700 con 0x55d891c6e1a0 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d891b5a810 con 0x55d891c7a130 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47652/0 (socket says 172.21.15.107:47652) 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 -- 172.21.15.107:0/3501077901 learned_addr learned my addr 172.21.15.107:0/3501077901 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6d880 0x55d891c77150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d891c81190 con 0x55d891c6d880 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891c77bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37b6e3c740 10 client.0 _ll_register_callbacks cb 0x55d891bcd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 -- 172.21.15.107:0/3501077901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6d880 msgr2=0x55d891c77150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6d880 0x55d891c77150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.101 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 -- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 msgr2=0x55d891c77bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891c77bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.093+0000 7f37add9b640 1 -- 172.21.15.107:0/3501077901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d891c8a5c0 con 0x55d891c6e1a0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891c77bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f37add9b640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f37a4012c20 tx=0x7f37a40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834de776d1f8ddfb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f3795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a401f060 con 0x55d891c6e1a0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37a400e7c0 con 0x55d891c6e1a0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d891b83150 con 0x55d891c6e1a0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37a400e9c0 con 0x55d891c6e1a0 2024-08-20T06:38:28.102 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f37b6e3c740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.094+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d891c8cee0 con 0x55d891c6e1a0 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d891b83150 con 0x55d891c6e1a0 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37a4024070 con 0x55d891c6e1a0 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 20 client.15618 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c8e3f0 0x55d891c90830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 4 client.15618 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:28.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d891be6880 con 0x55d891c8e3f0 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.095+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c8e3f0 0x55d891c90830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.096+0000 7f3795ffb640 10 client.15618 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.096+0000 7f37aed9d640 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c8e3f0 0x55d891c90830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=660ecef51430e2c0 server_cookie=a724cca1a55c83d2 in_seq=0 out_seq=0 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.096+0000 7f3795ffb640 1 -- 172.21.15.107:0/3501077901 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d891be6880 con 0x55d891c8e3f0 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.096+0000 7f3795ffb640 10 client.15618 handle_command_reply: tid=0 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 client.15618 shutdown 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 msgr2=0x55d891c77690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f37a4012c20 tx=0x7f37a40107a0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 shutdown_connections 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891c77bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c77690 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c8e3f0 0x55d891c90830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/3501077901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6d880 0x55d891c77150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 >> 172.21.15.107:0/3501077901 conn(0x55d891c6b6b0 msgr2=0x55d891c71250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f3795ffb640 0 client.15618 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 shutdown_connections 2024-08-20T06:38:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 1 -- 172.21.15.107:0/3501077901 wait complete. 2024-08-20T06:38:28.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.097+0000 7f37b6e3c740 20 client.15618 trim_cache size 0 max 16384 2024-08-20T06:38:28.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.116+0000 7f37b6e3c740 1 Processor -- start 2024-08-20T06:38:28.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.116+0000 7f37b6e3c740 1 -- start start 2024-08-20T06:38:28.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.116+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c78780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.116+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c96640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47658/0 (socket says 172.21.15.107:47658) 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 -- 172.21.15.107:0/2369441334 learned_addr learned my addr 172.21.15.107:0/2369441334 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b6fd0 con 0x55d891c6e1a0 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b48a0 con 0x55d891c6d880 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b26f0 con 0x55d891c7a130 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37aed9d640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c78780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37aed9d640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c96640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 -- 172.21.15.107:0/2369441334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 msgr2=0x55d891c96640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c96640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 -- 172.21.15.107:0/2369441334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 msgr2=0x55d891c78780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c78780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 -- 172.21.15.107:0/2369441334 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d891c66830 con 0x55d891c6e1a0 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37aed9d640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c96640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:28.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.117+0000 7f37add9b640 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a40056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26a1031d8f60f3f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.118+0000 7f3797fff640 1 -- 172.21.15.107:0/2369441334 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a4033070 con 0x55d891c6e1a0 2024-08-20T06:38:28.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.118+0000 7f3797fff640 1 -- 172.21.15.107:0/2369441334 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a400e7c0 con 0x55d891c6e1a0 2024-08-20T06:38:28.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.118+0000 7f3797fff640 1 -- 172.21.15.107:0/2369441334 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a402a070 con 0x55d891c6e1a0 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.118+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 msgr2=0x55d891c96100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.118+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000c00 tx=0x7f37a40056f0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 shutdown_connections 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c7a130 0x55d891c96640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6e1a0 0x55d891c96100 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/2369441334 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c6d880 0x55d891c78780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 >> 172.21.15.107:0/2369441334 conn(0x55d891c6b6b0 msgr2=0x55d891c6cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 shutdown_connections 2024-08-20T06:38:28.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 -- 172.21.15.107:0/2369441334 wait complete. 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 -- start start 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 Processor -- start 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.119+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 0x55d891c84020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891cf70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b2d60 con 0x55d891c6d880 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47674/0 (socket says 172.21.15.107:47674) 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b41d0 con 0x55d891c7a130 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 -- 172.21.15.107:0/141019688 learned_addr learned my addr 172.21.15.107:0/141019688 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f37b01b3430 con 0x55d891c6e1a0 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37b6e3c740 10 client.0 _ll_register_callbacks cb 0x7f37a400aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891cf70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 -- 172.21.15.107:0/141019688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 msgr2=0x55d891c84020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37add9b640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 0x55d891c84020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 0x55d891c84020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 -- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 msgr2=0x55d891cf70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891cf70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d891cf77c0 con 0x55d891c6d880 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37add9b640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 0x55d891c84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f37aed9d640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f37a8002230 tx=0x7f37a80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a61e8ec3ab6dfa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.120+0000 7f3795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a804a040 con 0x55d891c6d880 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f37a8003bb0 con 0x55d891c6d880 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37a400e7c0 con 0x55d891c6d880 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37a8004b40 con 0x55d891c6d880 2024-08-20T06:38:28.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f37b6e3c740 10 client.15624 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:28.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.121+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d891c7ce80 con 0x55d891c6d880 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f37a400e7c0 con 0x55d891c6d880 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f37a8061030 con 0x55d891c6d880 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 10 client.15624 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 20 client.15624 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c7d210 0x55d891c7f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 4 client.15624 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d891be2320 con 0x55d891c7d210 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f37add9b640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c7d210 0x55d891c7f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.122+0000 7f3795ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37add9b640 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c7d210 0x55d891c7f630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da4f02e7897a630 server_cookie=c04a9c11c371674a in_seq=0 out_seq=0 2024-08-20T06:38:28.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f3795ffb640 1 -- 172.21.15.107:0/141019688 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d891be2320 con 0x55d891c7d210 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f3795ffb640 10 client.15624 handle_command_reply: tid=0 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 client.15624 shutdown 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 msgr2=0x55d891c83ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f37a8002230 tx=0x7f37a80040e0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 shutdown_connections 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d891c7a130 0x55d891cf70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d891c7d210 0x55d891c7f630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d891c6e1a0 0x55d891c84020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 --2- 172.21.15.107:0/141019688 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d891c6d880 0x55d891c83ae0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f3795ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.123+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 >> 172.21.15.107:0/141019688 conn(0x55d891c6b6b0 msgr2=0x55d891c6c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 shutdown_connections 2024-08-20T06:38:28.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b6e3c740 1 -- 172.21.15.107:0/141019688 wait complete. 2024-08-20T06:38:28.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b6e3c740 20 client.15624 trim_cache size 0 max 16384 2024-08-20T06:38:28.129 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:28.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 msgr2=0x7f37980b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 0x7f37980b6090 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f37a00a7050 tx=0x7f37a0010ce0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 msgr2=0x7f37b00744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.124+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37a000e9f0 tx=0x7f37a000edf0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 shutdown_connections 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f37980b3bd0 0x7f37980b6090 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37b0107d20 0x7f37b006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f37b01073a0 0x7f37b0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 --2- 172.21.15.107:0/215823811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37b0101990 0x7f37b00744c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 >> 172.21.15.107:0/215823811 conn(0x7f37b00fd400 msgr2=0x7f37b00fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 shutdown_connections 2024-08-20T06:38:28.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:28.125+0000 7f37b4a43640 1 -- 172.21.15.107:0/215823811 wait complete. 2024-08-20T06:38:28.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:28 smithi183 ceph-mon[22204]: pgmap v376: 97 pgs: 97 active+clean; 46 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-08-20T06:38:29.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:28 smithi193 ceph-mon[21944]: pgmap v376: 97 pgs: 97 active+clean; 46 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-08-20T06:38:29.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:28 smithi107 ceph-mon[18495]: pgmap v376: 97 pgs: 97 active+clean; 46 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.6 MiB/s wr, 106 op/s 2024-08-20T06:38:29.897 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:38:29.898 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:38:30.247 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:30.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 -- 172.21.15.107:0/1028358670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341073a0 msgr2=0x7f45341077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 --2- 172.21.15.107:0/1028358670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341073a0 0x7f45341077e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f452402a410 comp rx=0 tx=0).stop 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 -- 172.21.15.107:0/1028358670 shutdown_connections 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 --2- 172.21.15.107:0/1028358670 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4534107d20 0x7f453410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 --2- 172.21.15.107:0/1028358670 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341073a0 0x7f45341077e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 --2- 172.21.15.107:0/1028358670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534101990 0x7f4534101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 -- 172.21.15.107:0/1028358670 >> 172.21.15.107:0/1028358670 conn(0x7f45340fd440 msgr2=0x7f45340ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 -- 172.21.15.107:0/1028358670 shutdown_connections 2024-08-20T06:38:30.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.841+0000 7f453bacd640 1 -- 172.21.15.107:0/1028358670 wait complete. 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.842+0000 7f453bacd640 1 Processor -- start 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.842+0000 7f453bacd640 1 -- start start 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.842+0000 7f453bacd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.842+0000 7f453bacd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 0x7f453410dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.842+0000 7f453bacd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 0x7f45341a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f453bacd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112990 con 0x7f4534101990 2024-08-20T06:38:30.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f453bacd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112810 con 0x7f45341073a0 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f453bacd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112b10 con 0x7f4534107d20 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47702/0 (socket says 172.21.15.107:47702) 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 -- 172.21.15.107:0/3529977621 learned_addr learned my addr 172.21.15.107:0/3529977621 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539068640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 0x7f453410dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f453a06a640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 0x7f45341a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 -- 172.21.15.107:0/3529977621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 msgr2=0x7f45341a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 0x7f45341a1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 -- 172.21.15.107:0/3529977621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 msgr2=0x7f453410dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 0x7f453410dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 -- 172.21.15.107:0/3529977621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45341a23f0 con 0x7f4534101990 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f453a06a640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 0x7f45341a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539068640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 0x7f453410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:30.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.843+0000 7f4539869640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f453000e9f0 tx=0x7f453000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.844+0000 7f452affd640 1 -- 172.21.15.107:0/3529977621 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f453000cd80 con 0x7f4534101990 2024-08-20T06:38:30.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.844+0000 7f452affd640 1 -- 172.21.15.107:0/3529977621 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45300054b0 con 0x7f4534101990 2024-08-20T06:38:30.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.844+0000 7f452affd640 1 -- 172.21.15.107:0/3529977621 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4530010430 con 0x7f4534101990 2024-08-20T06:38:30.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.844+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45341a26e0 con 0x7f4534101990 2024-08-20T06:38:30.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.844+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341abfe0 con 0x7f4534101990 2024-08-20T06:38:30.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.847+0000 7f452affd640 1 -- 172.21.15.107:0/3529977621 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f45300046c0 con 0x7f4534101990 2024-08-20T06:38:30.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.847+0000 7f453dec6740 1 Processor -- start 2024-08-20T06:38:30.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.848+0000 7f453dec6740 1 -- start start 2024-08-20T06:38:30.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.848+0000 7f452affd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 0x7f45200b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.848+0000 7f453a06a640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 0x7f45200b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.848+0000 7f452affd640 1 -- 172.21.15.107:0/3529977621 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f45300e3fa0 con 0x7f4534101990 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x563241472790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x563241481050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5632413e8940 con 0x563241472cd0 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5632413d7e40 con 0x56324147ec60 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453dec6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5632413dbe80 con 0x5632414723b0 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47716/0 (socket says 172.21.15.107:47716) 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 -- 172.21.15.107:0/2003418684 learned_addr learned my addr 172.21.15.107:0/2003418684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539869640 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x563241472790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f453a06a640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 0x7f45200b6110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 -- 172.21.15.107:0/2003418684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 msgr2=0x563241472790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x563241472790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 -- 172.21.15.107:0/2003418684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 msgr2=0x563241481050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x563241481050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.849+0000 7f4539068640 1 -- 172.21.15.107:0/2003418684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563241374520 con 0x563241472cd0 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.850+0000 7f4539068640 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f4524032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24365df049f5381 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.850+0000 7f451ffff640 1 -- 172.21.15.107:0/2003418684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524035070 con 0x563241472cd0 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.850+0000 7f451ffff640 1 -- 172.21.15.107:0/2003418684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45240044c0 con 0x563241472cd0 2024-08-20T06:38:30.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.850+0000 7f451ffff640 1 -- 172.21.15.107:0/2003418684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524003cd0 con 0x563241472cd0 2024-08-20T06:38:30.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 -- 172.21.15.107:0/2003418684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 msgr2=0x56324147e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f4524032e30 comp rx=0 tx=0).stop 2024-08-20T06:38:30.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 -- 172.21.15.107:0/2003418684 shutdown_connections 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x563241481050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147e720 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 --2- 172.21.15.107:0/2003418684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x563241472790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 -- 172.21.15.107:0/2003418684 >> 172.21.15.107:0/2003418684 conn(0x5632414701e0 msgr2=0x563241471380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 -- 172.21.15.107:0/2003418684 shutdown_connections 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.851+0000 7f453dec6740 1 -- 172.21.15.107:0/2003418684 wait complete. 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.852+0000 7f453dec6740 1 -- start start 2024-08-20T06:38:30.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.852+0000 7f453dec6740 1 Processor -- start 2024-08-20T06:38:30.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.853+0000 7f453dec6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.853+0000 7f453dec6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.853+0000 7f453dec6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x56324147c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539869640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47724/0 (socket says 172.21.15.107:47724) 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539869640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:60228/0 (socket says 172.21.15.107:60228) 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539068640 1 -- 172.21.15.107:0/1367721489 learned_addr learned my addr 172.21.15.107:0/1367721489 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56324123a890 con 0x563241472cd0 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56324135e700 con 0x56324147ec60 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x56324147c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563241481950 con 0x5632414723b0 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453dec6740 10 client.0 _ll_register_callbacks cb 0x5632413d36f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 -- 172.21.15.107:0/1367721489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 msgr2=0x56324147bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 -- 172.21.15.107:0/1367721489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 msgr2=0x56324147c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563241137930 con 0x56324147ec60 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f4539068640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f453a06a640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x56324147c6b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f452c00fbc0 tx=0x7f452c011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1299df581e012f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.854+0000 7f451d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c022020 con 0x56324147ec60 2024-08-20T06:38:30.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f452c003d80 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632413ea890 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f452c004050 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f453dec6740 10 client.25142 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56324148d590 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5632413ea890 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f452c010a00 con 0x56324147ec60 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f453dec6740 10 client.25142 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.855+0000 7f453dec6740 10 client.25142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f453dec6740 20 client.25142 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56324148eb40 0x563241490f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f453dec6740 4 client.25142 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:30.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632413e6330 con 0x56324148eb40 2024-08-20T06:38:30.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f4539869640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56324148eb40 0x563241490f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f451d7fa640 10 client.25142 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:30.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.856+0000 7f4539869640 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56324148eb40 0x563241490f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fe5d2951343531 server_cookie=f57ed0725b56a68d in_seq=0 out_seq=0 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f451d7fa640 1 -- 172.21.15.107:0/1367721489 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5632413e6330 con 0x56324148eb40 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f451d7fa640 10 client.25142 handle_command_reply: tid=0 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 client.25142 shutdown 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 msgr2=0x56324147c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x56324147c6b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f452c00fbc0 tx=0x7f452c011a90 comp rx=0 tx=0).stop 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 shutdown_connections 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56324147ec60 0x56324147c6b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563241472cd0 0x56324147c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56324148eb40 0x563241490f80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 --2- 172.21.15.107:0/1367721489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 >> 172.21.15.107:0/1367721489 conn(0x5632414701e0 msgr2=0x563241475d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f451d7fa640 0 client.25142 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 shutdown_connections 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 1 -- 172.21.15.107:0/1367721489 wait complete. 2024-08-20T06:38:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.858+0000 7f453dec6740 20 client.25142 trim_cache size 0 max 16384 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 Processor -- start 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 -- start start 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 0x563241496d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b6fb0 con 0x56324147ec60 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b4880 con 0x563241472cd0 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.882+0000 7f453dec6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b26b0 con 0x5632414723b0 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f4539068640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f4539068640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48318/0 (socket says 172.21.15.107:48318) 2024-08-20T06:38:30.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f4539068640 1 -- 172.21.15.107:0/382797810 learned_addr learned my addr 172.21.15.107:0/382797810 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 0x563241496d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f4539869640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 -- 172.21.15.107:0/382797810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 msgr2=0x56324147c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 -- 172.21.15.107:0/382797810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 msgr2=0x563241496850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 -- 172.21.15.107:0/382797810 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56324147ab90 con 0x56324147ec60 2024-08-20T06:38:30.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f4539068640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.883+0000 7f453a06a640 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 0x563241496d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f452c010ef0 tx=0x7f452c009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bb2ec8d10f28b4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.884+0000 7f451f7fe640 1 -- 172.21.15.107:0/382797810 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c036070 con 0x56324147ec60 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.884+0000 7f451f7fe640 1 -- 172.21.15.107:0/382797810 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f452c0041f0 con 0x56324147ec60 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.884+0000 7f451f7fe640 1 -- 172.21.15.107:0/382797810 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c022040 con 0x56324147ec60 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- 172.21.15.107:0/382797810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 msgr2=0x563241496d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 0x563241496d90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f452c010ef0 tx=0x7f452c009cf0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- 172.21.15.107:0/382797810 shutdown_connections 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56324147ec60 0x563241496d90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241496850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 --2- 172.21.15.107:0/382797810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5632414723b0 0x56324147c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- 172.21.15.107:0/382797810 >> 172.21.15.107:0/382797810 conn(0x5632414701e0 msgr2=0x56324147bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- 172.21.15.107:0/382797810 shutdown_connections 2024-08-20T06:38:30.887 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- 172.21.15.107:0/382797810 wait complete. 2024-08-20T06:38:30.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 -- start start 2024-08-20T06:38:30.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.885+0000 7f453dec6740 1 Processor -- start 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 0x563241482af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241483030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56324147ec60 0x563241483570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b2d40 con 0x5632414723b0 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b41b0 con 0x563241472cd0 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45341b3410 con 0x56324147ec60 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453dec6740 10 client.0 _ll_register_callbacks cb 0x7f452c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539869640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 0x563241482af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539869640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 0x563241482af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47740/0 (socket says 172.21.15.107:47740) 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241483030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539869640 1 -- 172.21.15.107:0/2205403828 learned_addr learned my addr 172.21.15.107:0/2205403828 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f453a06a640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56324147ec60 0x563241483570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 -- 172.21.15.107:0/2205403828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56324147ec60 msgr2=0x563241483570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56324147ec60 0x563241483570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 -- 172.21.15.107:0/2205403828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 msgr2=0x563241482af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 0x563241482af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.886+0000 7f4539068640 1 -- 172.21.15.107:0/2205403828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632414f88d0 con 0x563241472cd0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f4539068640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241483030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f4524032fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78cdce35bc0501c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f451d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524054060 con 0x563241472cd0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452c0041f0 con 0x563241472cd0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4524003c50 con 0x563241472cd0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4524035040 con 0x563241472cd0 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f453dec6740 10 client.25148 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:30.891 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5632414f9c60 con 0x563241472cd0 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.887+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f452c0041f0 con 0x563241472cd0 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4524059070 con 0x563241472cd0 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 10 client.25148 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 10 client.25148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 20 client.25148 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5632414f9f60 0x5632414fc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:30.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 4 client.25148 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:38:30.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5632414769d0 con 0x5632414f9f60 2024-08-20T06:38:30.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f4539869640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5632414f9f60 0x5632414fc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:30.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f451d7fa640 10 client.25148 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:30.893 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.888+0000 7f4539869640 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5632414f9f60 0x5632414fc380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d1e39f5a2676bb2 server_cookie=28870fefd432d699 in_seq=0 out_seq=0 2024-08-20T06:38:30.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f451d7fa640 1 -- 172.21.15.107:0/2205403828 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5632414769d0 con 0x5632414f9f60 2024-08-20T06:38:30.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f451d7fa640 10 client.25148 handle_command_reply: tid=0 2024-08-20T06:38:30.894 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 client.25148 shutdown 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 msgr2=0x563241483030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241483030 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4524009900 tx=0x7f4524032fc0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 shutdown_connections 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5632414f9f60 0x5632414fc380 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56324147ec60 0x563241483570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563241472cd0 0x563241483030 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f451d7fa640 0 client.25148 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 --2- 172.21.15.107:0/2205403828 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5632414723b0 0x563241482af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 >> 172.21.15.107:0/2205403828 conn(0x5632414701e0 msgr2=0x563241475c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.890+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 shutdown_connections 2024-08-20T06:38:30.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453dec6740 1 -- 172.21.15.107:0/2205403828 wait complete. 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453dec6740 20 client.25148 trim_cache size 0 max 16384 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "c8d7ec9a-4a66-4e44-b463-ec8103237292", 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:38:30.896 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 msgr2=0x7f45200b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 0x7f45200b6110 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f452c005fd0 tx=0x7f452c009500 comp rx=0 tx=0).stop 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 msgr2=0x7f453410d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f453000e9f0 tx=0x7f453000edf0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 shutdown_connections 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45200b3c50 0x7f45200b6110 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4534107d20 0x7f45341a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45341073a0 0x7f453410dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.891+0000 7f453bacd640 1 --2- 172.21.15.107:0/3529977621 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4534101990 0x7f453410d820 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:30.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.892+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 >> 172.21.15.107:0/3529977621 conn(0x7f45340fd440 msgr2=0x7f4534103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:30.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.892+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 shutdown_connections 2024-08-20T06:38:30.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:30.892+0000 7f453bacd640 1 -- 172.21.15.107:0/3529977621 wait complete. 2024-08-20T06:38:30.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:30 smithi183 ceph-mon[22204]: pgmap v377: 97 pgs: 97 active+clean; 47 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-08-20T06:38:31.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:30 smithi193 ceph-mon[21944]: pgmap v377: 97 pgs: 97 active+clean; 47 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-08-20T06:38:31.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:30 smithi107 ceph-mon[18495]: pgmap v377: 97 pgs: 97 active+clean; 47 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.2 MiB/s wr, 93 op/s 2024-08-20T06:38:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:31 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:38:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:31 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:31 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:38:31.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:31 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:32.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:31 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:38:32.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:31 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:32.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:31 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:38:32.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:31 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:31 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:38:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:31 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:31 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:38:32.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:31 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:32.575 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:38:32.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:32 smithi107 ceph-mon[18495]: pgmap v378: 97 pgs: 97 active+clean; 48 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-08-20T06:38:32.945 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:32.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:32 smithi183 ceph-mon[22204]: pgmap v378: 97 pgs: 97 active+clean; 48 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-08-20T06:38:33.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:32 smithi193 ceph-mon[21944]: pgmap v378: 97 pgs: 97 active+clean; 48 MiB data, 557 MiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 1.2 MiB/s wr, 92 op/s 2024-08-20T06:38:34.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:34 smithi183 ceph-mon[22204]: pgmap v379: 97 pgs: 97 active+clean; 53 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:38:35.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:34 smithi193 ceph-mon[21944]: pgmap v379: 97 pgs: 97 active+clean; 53 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:38:35.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:34 smithi107 ceph-mon[18495]: pgmap v379: 97 pgs: 97 active+clean; 53 MiB data, 574 MiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:38:35.884 INFO:tasks.workunit.client.0.smithi107.stdout: 11 13250 163 10272 128 4672 313 2024-08-20T06:38:35.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:35 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:38:36.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:35 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:38:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:35 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:38:36.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:36 smithi183 ceph-mon[22204]: pgmap v380: 97 pgs: 97 active+clean; 54 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-20T06:38:37.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:36 smithi193 ceph-mon[21944]: pgmap v380: 97 pgs: 97 active+clean; 54 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-20T06:38:37.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:36 smithi107 ceph-mon[18495]: pgmap v380: 97 pgs: 97 active+clean; 54 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 98 op/s 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.307+0000 7fc3b5b75640 1 -- 172.21.15.107:0/1730507826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0102af0 msgr2=0x7fc3b0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/1730507826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0102af0 0x7fc3b0102ed0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0009900 tx=0x7fc3a002a410 comp rx=0 tx=0).stop 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 -- 172.21.15.107:0/1730507826 shutdown_connections 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/1730507826 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 0x7fc3b010f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/1730507826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0108500 0x7fc3b0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/1730507826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0102af0 0x7fc3b0102ed0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 -- 172.21.15.107:0/1730507826 >> 172.21.15.107:0/1730507826 conn(0x7fc3b00fe540 msgr2=0x7fc3b0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 -- 172.21.15.107:0/1730507826 shutdown_connections 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 -- 172.21.15.107:0/1730507826 wait complete. 2024-08-20T06:38:37.212 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 Processor -- start 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.308+0000 7fc3b5b75640 1 -- start start 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0102af0 0x7fc3b0198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 0x7fc3b019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01129b0 con 0x7fc3b0108500 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0112830 con 0x7fc3b0108e80 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b0112b30 con 0x7fc3b0102af0 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55246/0 (socket says 172.21.15.107:55246) 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 -- 172.21.15.107:0/2344357800 learned_addr learned my addr 172.21.15.107:0/2344357800 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0102af0 0x7fc3b0198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3affff640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 0x7fc3b019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 -- 172.21.15.107:0/2344357800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0102af0 msgr2=0x7fc3b0198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0102af0 0x7fc3b0198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 -- 172.21.15.107:0/2344357800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 msgr2=0x7fc3b019e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 0x7fc3b019e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 -- 172.21.15.107:0/2344357800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b019ef50 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400d950 tx=0x7fc3a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3acff9640 1 -- 172.21.15.107:0/2344357800 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a4017030 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3acff9640 1 -- 172.21.15.107:0/2344357800 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3a4004020 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b019f240 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3acff9640 1 -- 172.21.15.107:0/2344357800 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40055d0 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.309+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b01a5900 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3acff9640 1 -- 172.21.15.107:0/2344357800 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fc3a4009a30 con 0x7fc3b0108500 2024-08-20T06:38:37.214 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 Processor -- start 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 -- start start 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d930c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d93cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 0x560c7d93f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c7d8a78e0 con 0x560c7d93d0e0 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560c7d896de0 con 0x560c7d930830 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c7d89ae20 con 0x560c7d931150 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.312+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 0x560c7d93f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d93cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 0x560c7d93f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55262/0 (socket says 172.21.15.107:55262) 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3aeffd640 1 -- 172.21.15.107:0/4203986848 learned_addr learned my addr 172.21.15.107:0/4203986848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3affff640 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d930c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.215 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 -- 172.21.15.107:0/4203986848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 msgr2=0x560c7d930c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d930c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 -- 172.21.15.107:0/4203986848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 msgr2=0x560c7d93f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 0x560c7d93f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 -- 172.21.15.107:0/4203986848 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7d5f6650 con 0x560c7d931150 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3acff9640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 0x7fc3980b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d93cba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005940 tx=0x7fc3a0031bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5df68fae55097e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3affff640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 0x7fc3980b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc3acff9640 1 -- 172.21.15.107:0/2344357800 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fc3a40e35c0 con 0x7fc3b0108500 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc395ffb640 1 -- 172.21.15.107:0/4203986848 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0035070 con 0x560c7d931150 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc395ffb640 1 -- 172.21.15.107:0/4203986848 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3a0040070 con 0x560c7d931150 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.313+0000 7fc395ffb640 1 -- 172.21.15.107:0/4203986848 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0003ae0 con 0x560c7d931150 2024-08-20T06:38:37.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3affff640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 0x7fc3980b6090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8005fd0 tx=0x7fc3a8009600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/4203986848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 msgr2=0x560c7d93cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d93cba0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005940 tx=0x7fc3a0031bc0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/4203986848 shutdown_connections 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d93d0e0 0x560c7d93f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d93cba0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/4203986848 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d930c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/4203986848 >> 172.21.15.107:0/4203986848 conn(0x560c7d92e660 msgr2=0x560c7d92f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/4203986848 shutdown_connections 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/4203986848 wait complete. 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.314+0000 7fc3b7f6e740 1 -- start start 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 Processor -- start 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.217 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d931150 0x560c7d93a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x560c7d6f9700 con 0x560c7d930830 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560c7d81d7c0 con 0x560c7d931150 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560c7d93fd30 con 0x560c7d93d0e0 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3b7f6e740 10 client.0 _ll_register_callbacks cb 0x560c7d8906b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55268/0 (socket says 172.21.15.107:55268) 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 -- 172.21.15.107:0/570400326 learned_addr learned my addr 172.21.15.107:0/570400326 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d931150 0x560c7d93a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3affff640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 -- 172.21.15.107:0/570400326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 msgr2=0x560c7d93aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 -- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d931150 msgr2=0x560c7d93a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d931150 0x560c7d93a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.315+0000 7fc3af7fe640 1 -- 172.21.15.107:0/570400326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7d9490e0 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0031770 tx=0x7fc3a00317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcd7f13b96f0e97 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a004c020 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fc3a0004240 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560c7d846100 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3a0035070 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 10 client.15654 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560c7d94ba90 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x560c7d846100 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3a00454d0 con 0x560c7d930830 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 10 client.15654 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:37.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 20 client.15654 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d94cde0 0x560c7d94f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 4 client.15654 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.316+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c7d8a9830 con 0x560c7d94cde0 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.317+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d94cde0 0x560c7d94f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.317+0000 7fc38effd640 10 client.15654 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.317+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d94cde0 0x560c7d94f1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31bdb94ef6abb428 server_cookie=8cbd0c9f6c1da083 in_seq=0 out_seq=0 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc38effd640 1 -- 172.21.15.107:0/570400326 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c7d8a9830 con 0x560c7d94cde0 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc38effd640 10 client.15654 handle_command_reply: tid=0 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 client.15654 shutdown 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 msgr2=0x560c7d93a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0031770 tx=0x7fc3a00317a0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 shutdown_connections 2024-08-20T06:38:37.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d94cde0 0x560c7d94f1d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d931150 0x560c7d93a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc38effd640 0 client.15654 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/570400326 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d93a070 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 >> 172.21.15.107:0/570400326 conn(0x560c7d92e660 msgr2=0x560c7d934200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 shutdown_connections 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/570400326 wait complete. 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.318+0000 7fc3b7f6e740 20 client.15654 trim_cache size 0 max 16384 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.335+0000 7fc3b7f6e740 1 Processor -- start 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.335+0000 7fc3b7f6e740 1 -- start start 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d934930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01b2b40 con 0x560c7d931150 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01b0410 con 0x560c7d930830 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01ae240 con 0x560c7d93d0e0 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55272/0 (socket says 172.21.15.107:55272) 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 -- 172.21.15.107:0/1756952438 learned_addr learned my addr 172.21.15.107:0/1756952438 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d934930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3affff640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 -- 172.21.15.107:0/1756952438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 msgr2=0x560c7d93aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 -- 172.21.15.107:0/1756952438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 msgr2=0x560c7d934930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d934930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 -- 172.21.15.107:0/1756952438 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c7d9301f0 con 0x560c7d931150 2024-08-20T06:38:37.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d934930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005ad0 tx=0x7fc3a40cd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b41c19f271704b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3957fa640 1 -- 172.21.15.107:0/1756952438 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40a9e60 con 0x560c7d931150 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3957fa640 1 -- 172.21.15.107:0/1756952438 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3a40cd0b0 con 0x560c7d931150 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.336+0000 7fc3957fa640 1 -- 172.21.15.107:0/1756952438 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a40a9c80 con 0x560c7d931150 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/1756952438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 msgr2=0x560c7d93a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4005ad0 tx=0x7fc3a40cd040 comp rx=0 tx=0).stop 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/1756952438 shutdown_connections 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d93d0e0 0x560c7d93aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d931150 0x560c7d93a570 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/1756952438 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d930830 0x560c7d934930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/1756952438 >> 172.21.15.107:0/1756952438 conn(0x560c7d92e660 msgr2=0x560c7d93bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/1756952438 shutdown_connections 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/1756952438 wait complete. 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.337+0000 7fc3b7f6e740 1 -- start start 2024-08-20T06:38:37.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 Processor -- start 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d9408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 0x560c7d940df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01ae8d0 con 0x560c7d930830 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01afd40 con 0x560c7d93d0e0 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b01aefa0 con 0x560c7d931150 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3b7f6e740 10 client.0 _ll_register_callbacks cb 0x7fc3a40e6f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55286/0 (socket says 172.21.15.107:55286) 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 -- 172.21.15.107:0/127847891 learned_addr learned my addr 172.21.15.107:0/127847891 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3affff640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 0x560c7d940df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d9408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 -- 172.21.15.107:0/127847891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 msgr2=0x560c7d9408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d9408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 -- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 msgr2=0x560c7d940df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:37.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 0x560c7d940df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 -- 172.21.15.107:0/127847891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c7d938fd0 con 0x560c7d930830 2024-08-20T06:38:37.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d9408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:37.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3affff640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 0x560c7d940df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc3af7fe640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005c40 tx=0x7fc3a0031a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb64189c642aca server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.338+0000 7fc38effd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a004c040 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fc3a0045460 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3a40cd0b0 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc3a0045660 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 10 client.15660 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560c7d9b73d0 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fc3a40cd0b0 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc3a0065020 con 0x560c7d930830 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 10 client.15660 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 10 client.15660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:37.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 20 client.15660 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d9b7b50 0x560c7d9b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 4 client.15660 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560c7d8a52d0 con 0x560c7d9b7b50 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.339+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d9b7b50 0x560c7d9b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc38effd640 10 client.15660 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3aeffd640 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d9b7b50 0x560c7d9b9f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f796edd3d0eb47 server_cookie=a399f7f594670278 in_seq=0 out_seq=0 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc38effd640 1 -- 172.21.15.107:0/127847891 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560c7d8a52d0 con 0x560c7d9b7b50 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc38effd640 10 client.15660 handle_command_reply: tid=0 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 client.15660 shutdown 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 msgr2=0x560c7d9453e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc3a0005c40 tx=0x7fc3a0031a40 comp rx=0 tx=0).stop 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 shutdown_connections 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560c7d93d0e0 0x560c7d940df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560c7d9b7b50 0x560c7d9b9f70 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560c7d931150 0x560c7d9408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 --2- 172.21.15.107:0/127847891 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560c7d930830 0x560c7d9453e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc38effd640 0 client.15660 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 >> 172.21.15.107:0/127847891 conn(0x560c7d92e660 msgr2=0x560c7d947e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 shutdown_connections 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 1 -- 172.21.15.107:0/127847891 wait complete. 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.340+0000 7fc3b7f6e740 20 client.15660 trim_cache size 0 max 16384 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 msgr2=0x7fc3980b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 0x7fc3980b6090 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8005fd0 tx=0x7fc3a8009600 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 msgr2=0x7fc3b01993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc3a400d950 tx=0x7fc3a400de10 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 shutdown_connections 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fc3b0108e80 0x7fc3b019e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3b0108500 0x7fc3b01993e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fc3980b3bd0 0x7fc3980b6090 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 --2- 172.21.15.107:0/2344357800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3b0102af0 0x7fc3b0198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 >> 172.21.15.107:0/2344357800 conn(0x7fc3b00fe540 msgr2=0x7fc3b0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 shutdown_connections 2024-08-20T06:38:37.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:36.341+0000 7fc3b5b75640 1 -- 172.21.15.107:0/2344357800 wait complete. 2024-08-20T06:38:38.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:38 smithi183 ceph-mon[22204]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-08-20T06:38:39.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:38 smithi193 ceph-mon[21944]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-08-20T06:38:39.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:38 smithi107 ceph-mon[18495]: pgmap v381: 97 pgs: 97 active+clean; 55 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.3 MiB/s wr, 112 op/s 2024-08-20T06:38:40.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:40 smithi183 ceph-mon[22204]: pgmap v382: 97 pgs: 97 active+clean; 59 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-20T06:38:41.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:40 smithi193 ceph-mon[21944]: pgmap v382: 97 pgs: 97 active+clean; 59 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-20T06:38:41.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:40 smithi107 ceph-mon[18495]: pgmap v382: 97 pgs: 97 active+clean; 59 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.4 MiB/s wr, 117 op/s 2024-08-20T06:38:42.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:42 smithi183 ceph-mon[22204]: pgmap v383: 97 pgs: 97 active+clean; 64 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-08-20T06:38:43.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:42 smithi193 ceph-mon[21944]: pgmap v383: 97 pgs: 97 active+clean; 64 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-08-20T06:38:43.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:42 smithi107 ceph-mon[18495]: pgmap v383: 97 pgs: 97 active+clean; 64 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.7 MiB/s wr, 128 op/s 2024-08-20T06:38:43.339 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:38:43.339 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:38:43.662 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:44.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:44 smithi183 ceph-mon[22204]: pgmap v384: 97 pgs: 97 active+clean; 68 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-08-20T06:38:45.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:44 smithi193 ceph-mon[21944]: pgmap v384: 97 pgs: 97 active+clean; 68 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-08-20T06:38:45.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:44 smithi107 ceph-mon[18495]: pgmap v384: 97 pgs: 97 active+clean; 68 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 1.9 MiB/s wr, 146 op/s 2024-08-20T06:38:45.885 INFO:tasks.workunit.client.0.smithi107.stdout: 16 13346 237 8865 215 3481 250 2024-08-20T06:38:46.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 -- 172.21.15.107:0/2879373829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 msgr2=0x7fef481077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:46.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 --2- 172.21.15.107:0/2879373829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef481077e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef38009900 tx=0x7fef3802a410 comp rx=0 tx=0).stop 2024-08-20T06:38:46.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 -- 172.21.15.107:0/2879373829 shutdown_connections 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 --2- 172.21.15.107:0/2879373829 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48107d20 0x7fef4810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 --2- 172.21.15.107:0/2879373829 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef481077e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 --2- 172.21.15.107:0/2879373829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48101990 0x7fef48101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.051+0000 7fef50457640 1 -- 172.21.15.107:0/2879373829 >> 172.21.15.107:0/2879373829 conn(0x7fef480fd400 msgr2=0x7fef480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.052+0000 7fef50457640 1 -- 172.21.15.107:0/2879373829 shutdown_connections 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.052+0000 7fef50457640 1 -- 172.21.15.107:0/2879373829 wait complete. 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.052+0000 7fef50457640 1 Processor -- start 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.052+0000 7fef50457640 1 -- start start 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48101990 0x7fef4819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:46.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48107d20 0x7fef481a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fef481129b0 con 0x7fef48101990 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fef48112830 con 0x7fef481073a0 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef50457640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fef48112b30 con 0x7fef48107d20 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48052/0 (socket says 172.21.15.107:48052) 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 -- 172.21.15.107:0/1105257324 learned_addr learned my addr 172.21.15.107:0/1105257324 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4e9f4640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48107d20 0x7fef481a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4e1f3640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48101990 0x7fef4819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 -- 172.21.15.107:0/1105257324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48107d20 msgr2=0x7fef481a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48107d20 0x7fef481a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 -- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48101990 msgr2=0x7fef4819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:46.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48101990 0x7fef4819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.053+0000 7fef4d9f2640 1 -- 172.21.15.107:0/1105257324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef481a1190 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef4d9f2640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef3802a930 tx=0x7fef38003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3803f070 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fef3803baf0 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38035030 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481a1420 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.054+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a1820 con 0x7fef481073a0 2024-08-20T06:38:46.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.055+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef48101d70 con 0x7fef481073a0 2024-08-20T06:38:46.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.057+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fef38042070 con 0x7fef481073a0 2024-08-20T06:38:46.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.058+0000 7fef3f7fe640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 0x7fef340b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:46.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.058+0000 7fef4e1f3640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 0x7fef340b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:46.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.058+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fef381041a0 con 0x7fef481073a0 2024-08-20T06:38:46.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.058+0000 7fef4e1f3640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 0x7fef340b6110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef4400bbe0 tx=0x7fef4400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:46.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.063+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fef380cb1a0 con 0x7fef481073a0 2024-08-20T06:38:46.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.413+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef4802e610 con 0x7fef481073a0 2024-08-20T06:38:46.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.414+0000 7fef3f7fe640 1 -- 172.21.15.107:0/1105257324 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11935 (secure 0 0 0) 0x7fef380cf6f0 con 0x7fef481073a0 2024-08-20T06:38:46.416 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:38:46.416 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":25,"btime":"2024-08-20T06:36:56:241712+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:36:56.009554+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:38:46.416 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 25 2024-08-20T06:38:46.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.417+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 msgr2=0x7fef340b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.417+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 0x7fef340b6110 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef4400bbe0 tx=0x7fef4400b390 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.417+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 msgr2=0x7fef4819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.417+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fef3802a930 tx=0x7fef38003b90 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 shutdown_connections 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fef340b3c50 0x7fef340b6110 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef48107d20 0x7fef481a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fef481073a0 0x7fef4819b620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 --2- 172.21.15.107:0/1105257324 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fef48101990 0x7fef4819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 >> 172.21.15.107:0/1105257324 conn(0x7fef480fd400 msgr2=0x7fef48104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 shutdown_connections 2024-08-20T06:38:46.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:46.418+0000 7fef50457640 1 -- 172.21.15.107:0/1105257324 wait complete. 2024-08-20T06:38:46.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:46 smithi183 ceph-mon[22204]: pgmap v385: 97 pgs: 97 active+clean; 69 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-08-20T06:38:46.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:46 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1105257324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:47.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:46 smithi193 ceph-mon[21944]: pgmap v385: 97 pgs: 97 active+clean; 69 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-08-20T06:38:47.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:46 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1105257324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:47.086 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:46 smithi107 ceph-mon[18495]: pgmap v385: 97 pgs: 97 active+clean; 69 MiB data, 648 MiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-08-20T06:38:47.087 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:46 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1105257324' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:38:47.123 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:38:47.479 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:48.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.073+0000 7f2b3ac78640 1 -- 172.21.15.107:0/1423791276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341084f0 msgr2=0x7f2b34108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.073+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/1423791276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341084f0 0x7f2b34108930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009900 tx=0x7f2b2002a3f0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.073+0000 7f2b3ac78640 1 -- 172.21.15.107:0/1423791276 shutdown_connections 2024-08-20T06:38:48.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.073+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/1423791276 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34108e70 0x7f2b3410f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.075 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.074+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/1423791276 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b341084f0 0x7f2b34108930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.074+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/1423791276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b34102ae0 0x7f2b34102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.074+0000 7f2b3ac78640 1 -- 172.21.15.107:0/1423791276 >> 172.21.15.107:0/1423791276 conn(0x7f2b340fe510 msgr2=0x7f2b34100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.074+0000 7f2b3ac78640 1 -- 172.21.15.107:0/1423791276 shutdown_connections 2024-08-20T06:38:48.076 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.074+0000 7f2b3ac78640 1 -- 172.21.15.107:0/1423791276 wait complete. 2024-08-20T06:38:48.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 Processor -- start 2024-08-20T06:38:48.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 -- start start 2024-08-20T06:38:48.077 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 0x7f2b34198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b341084f0 0x7f2b34199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34112a30 con 0x7f2b34108e70 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341128b0 con 0x7f2b34102ae0 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b3ac78640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34112bb0 con 0x7f2b341084f0 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b39215640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b39215640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46690/0 (socket says 172.21.15.107:46690) 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.075+0000 7f2b39215640 1 -- 172.21.15.107:0/2854535851 learned_addr learned my addr 172.21.15.107:0/2854535851 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 0x7f2b34198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b33fff640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b341084f0 0x7f2b34199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 -- 172.21.15.107:0/2854535851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b341084f0 msgr2=0x7f2b34199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.078 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b341084f0 0x7f2b34199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 -- 172.21.15.107:0/2854535851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 msgr2=0x7f2b34198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 0x7f2b34198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 -- 172.21.15.107:0/2854535851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3419f090 con 0x7f2b34108e70 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 0x7f2b34198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b39215640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00eb10 tx=0x7f2b2c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b31ffb640 1 -- 172.21.15.107:0/2854535851 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00ce20 con 0x7f2b34108e70 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3419f380 con 0x7f2b34108e70 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.076+0000 7f2b31ffb640 1 -- 172.21.15.107:0/2854535851 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c005020 con 0x7f2b34108e70 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.077+0000 7f2b31ffb640 1 -- 172.21.15.107:0/2854535851 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c010690 con 0x7f2b34108e70 2024-08-20T06:38:48.079 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.077+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341a5a40 con 0x7f2b34108e70 2024-08-20T06:38:48.080 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.079+0000 7f2b31ffb640 1 -- 172.21.15.107:0/2854535851 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f2b2c004a40 con 0x7f2b34108e70 2024-08-20T06:38:48.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.081+0000 7f2b31ffb640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 0x7f2b240b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.081+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 0x7f2b240b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.081+0000 7f2b31ffb640 1 -- 172.21.15.107:0/2854535851 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b2c0e4290 con 0x7f2b34108e70 2024-08-20T06:38:48.082 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.081+0000 7f2b3d071740 1 Processor -- start 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.081+0000 7f2b3d071740 1 -- start start 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2cedb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 0x5589f2cf9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5589f2c64930 con 0x5589f2cf9fe0 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589f2c53e30 con 0x5589f2cee050 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b3d071740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589f2c57e70 con 0x5589f2ced730 2024-08-20T06:38:48.083 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 0x7f2b240b6150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009d60 tx=0x7f2b28009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46702/0 (socket says 172.21.15.107:46702) 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 -- 172.21.15.107:0/3459153813 learned_addr learned my addr 172.21.15.107:0/3459153813 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b38a14640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2cedb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b33fff640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 0x5589f2cf9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 -- 172.21.15.107:0/3459153813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 msgr2=0x5589f2cedb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2cedb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 -- 172.21.15.107:0/3459153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 msgr2=0x5589f2cf9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.084 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 0x5589f2cf9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 -- 172.21.15.107:0/3459153813 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f29b3650 con 0x5589f2cf9fe0 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b33fff640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 0x5589f2cf9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b38a14640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2cedb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.082+0000 7f2b39215640 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aaa50 tx=0x7f2b2c0aaa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac361b8577bc02ed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.083+0000 7f2b1effd640 1 -- 172.21.15.107:0/3459153813 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0af7e0 con 0x5589f2cf9fe0 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.083+0000 7f2b1effd640 1 -- 172.21.15.107:0/3459153813 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b2c0b4290 con 0x5589f2cf9fe0 2024-08-20T06:38:48.085 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.083+0000 7f2b1effd640 1 -- 172.21.15.107:0/3459153813 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0aa440 con 0x5589f2cf9fe0 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.083+0000 7f2b3d071740 1 -- 172.21.15.107:0/3459153813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 msgr2=0x5589f2cfc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.083+0000 7f2b3d071740 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aaa50 tx=0x7f2b2c0aaa80 comp rx=0 tx=0).stop 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 -- 172.21.15.107:0/3459153813 shutdown_connections 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfc3d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cee050 0x5589f2cf9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.088 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 --2- 172.21.15.107:0/3459153813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2cedb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 -- 172.21.15.107:0/3459153813 >> 172.21.15.107:0/3459153813 conn(0x5589f2ceb560 msgr2=0x5589f2cec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 -- 172.21.15.107:0/3459153813 shutdown_connections 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 -- 172.21.15.107:0/3459153813 wait complete. 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 -- start start 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.084+0000 7f2b3d071740 1 Processor -- start 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cee050 0x5589f2cf7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cf7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5589f2ab6700 con 0x5589f2ced730 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5589f2bda810 con 0x5589f2cf9fe0 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5589f2cfcc90 con 0x5589f2cee050 2024-08-20T06:38:48.089 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b3d071740 10 client.0 _ll_register_callbacks cb 0x5589f2c4d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46718/0 (socket says 172.21.15.107:46718) 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 -- 172.21.15.107:0/13766995 learned_addr learned my addr 172.21.15.107:0/13766995 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b39215640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cf7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b33fff640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cee050 0x5589f2cf7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 -- 172.21.15.107:0/13766995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cee050 msgr2=0x5589f2cf7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cee050 0x5589f2cf7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 -- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 msgr2=0x5589f2cf7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cf7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 -- 172.21.15.107:0/13766995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f2d060c0 con 0x5589f2ced730 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b38a14640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009860 tx=0x7f2b280047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a601150b8a17af4b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.090 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.085+0000 7f2b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28024020 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b2800f040 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5589f2c03150 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b28015e60 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b3d071740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589f2d08a40 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5589f2c03150 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.086+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b28020c60 con 0x5589f2ced730 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 10 client.15678 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 20 client.15678 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:48.091 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d0c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 4 client.15678 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:48.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589f2c66880 con 0x5589f2d09f00 2024-08-20T06:38:48.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b33fff640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d0c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b33fff640 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d0c360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bff523ebee40f84f server_cookie=38f6b563362316fe in_seq=0 out_seq=0 2024-08-20T06:38:48.092 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.087+0000 7f2b13fff640 10 client.15678 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.088+0000 7f2b13fff640 1 -- 172.21.15.107:0/13766995 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5589f2c66880 con 0x5589f2d09f00 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.088+0000 7f2b13fff640 10 client.15678 handle_command_reply: tid=0 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.088+0000 7f2b3d071740 1 client.15678 shutdown 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.088+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 msgr2=0x5589f2cf7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.088+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009860 tx=0x7f2b280047d0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 shutdown_connections 2024-08-20T06:38:48.093 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d0c360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cf7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b13fff640 0 client.15678 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cee050 0x5589f2cf7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 --2- 172.21.15.107:0/13766995 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2ced730 0x5589f2cf7000 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 >> 172.21.15.107:0/13766995 conn(0x5589f2ceb560 msgr2=0x5589f2cf10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 shutdown_connections 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 1 -- 172.21.15.107:0/13766995 wait complete. 2024-08-20T06:38:48.094 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.089+0000 7f2b3d071740 20 client.15678 trim_cache size 0 max 16384 2024-08-20T06:38:48.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 Processor -- start 2024-08-20T06:38:48.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 -- start start 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d11c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 0x5589f2d12170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b2b90 con 0x5589f2cee050 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341b0460 con 0x5589f2ced730 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.112+0000 7f2b3d071740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341ae290 con 0x5589f2cf9fe0 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48096/0 (socket says 172.21.15.107:48096) 2024-08-20T06:38:48.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 -- 172.21.15.107:0/245152239 learned_addr learned my addr 172.21.15.107:0/245152239 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b33fff640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d11c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b39215640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 0x5589f2d12170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 -- 172.21.15.107:0/245152239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 msgr2=0x5589f2d12170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 0x5589f2d12170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 -- 172.21.15.107:0/245152239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 msgr2=0x5589f2d11c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d11c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 -- 172.21.15.107:0/245152239 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589f2cebfe0 con 0x5589f2ced730 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b33fff640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b39215640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 0x5589f2d12170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b38a14640 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009830 tx=0x7f2b28009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=717dc77eab7322b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.113+0000 7f2b1e7fc640 1 -- 172.21.15.107:0/245152239 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28024070 con 0x5589f2ced730 2024-08-20T06:38:48.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.114+0000 7f2b1e7fc640 1 -- 172.21.15.107:0/245152239 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b28013820 con 0x5589f2ced730 2024-08-20T06:38:48.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.114+0000 7f2b1e7fc640 1 -- 172.21.15.107:0/245152239 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2802bdf0 con 0x5589f2ced730 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 -- 172.21.15.107:0/245152239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 msgr2=0x5589f2cea3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009830 tx=0x7f2b28009ae0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 -- 172.21.15.107:0/245152239 shutdown_connections 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2cf9fe0 0x5589f2d12170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d11c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 --2- 172.21.15.107:0/245152239 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2ced730 0x5589f2cea3e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 -- 172.21.15.107:0/245152239 >> 172.21.15.107:0/245152239 conn(0x5589f2ceb560 msgr2=0x5589f2ce9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 -- 172.21.15.107:0/245152239 shutdown_connections 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.115+0000 7f2b3d071740 1 -- 172.21.15.107:0/245152239 wait complete. 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.116+0000 7f2b3d071740 1 -- start start 2024-08-20T06:38:48.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.116+0000 7f2b3d071740 1 Processor -- start 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2d022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d02830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341ae920 con 0x5589f2cee050 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b33fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d02830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2d022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b38a14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2d022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:54264/0 (socket says 172.21.15.107:54264) 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b38a14640 1 -- 172.21.15.107:0/2057191165 learned_addr learned my addr 172.21.15.107:0/2057191165 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b39215640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341afd90 con 0x5589f2cf9fe0 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.117+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341aeff0 con 0x5589f2ced730 2024-08-20T06:38:48.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b3d071740 10 client.0 _ll_register_callbacks cb 0x7f2b28006de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 -- 172.21.15.107:0/2057191165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 msgr2=0x5589f2d022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2d022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 -- 172.21.15.107:0/2057191165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 msgr2=0x5589f2d02830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d02830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589f2cfe230 con 0x5589f2cf9fe0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b33fff640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d02830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b39215640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfda00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aaee0 tx=0x7f2b2c0af920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab77c4b7da4dcf8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.118+0000 7f2b13fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c0c9070 con 0x5589f2cf9fe0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b2c004d30 con 0x5589f2cf9fe0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b28013820 con 0x5589f2cf9fe0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2b2c0cc360 con 0x5589f2cf9fe0 2024-08-20T06:38:48.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 10 client.25190 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:48.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5589f2d74910 con 0x5589f2cf9fe0 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b28013820 con 0x5589f2cf9fe0 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2b2c0a6b50 con 0x5589f2cf9fe0 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 10 client.25190 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 10 client.25190 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 20 client.25190 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d76c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 4 client.25190 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.119+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589f2c62320 con 0x5589f2d09f00 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d76c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b38a14640 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d76c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49ab623cfed0eecc server_cookie=1188a501fdf00c37 in_seq=0 out_seq=0 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b13fff640 10 client.25190 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:48.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b13fff640 1 -- 172.21.15.107:0/2057191165 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589f2c62320 con 0x5589f2d09f00 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b13fff640 10 client.25190 handle_command_reply: tid=0 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.120+0000 7f2b3d071740 1 client.25190 shutdown 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 msgr2=0x5589f2cfda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfda00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0aaee0 tx=0x7f2b2c0af920 comp rx=0 tx=0).stop 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 shutdown_connections 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x5589f2d09f00 0x5589f2d76c00 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5589f2cf9fe0 0x5589f2cfda00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5589f2cee050 0x5589f2d02830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b13fff640 0 client.25190 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 --2- 172.21.15.107:0/2057191165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589f2ced730 0x5589f2d022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 >> 172.21.15.107:0/2057191165 conn(0x5589f2ceb560 msgr2=0x5589f2cea4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 shutdown_connections 2024-08-20T06:38:48.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 1 -- 172.21.15.107:0/2057191165 wait complete. 2024-08-20T06:38:48.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.121+0000 7f2b3d071740 20 client.25190 trim_cache size 0 max 16384 2024-08-20T06:38:48.126 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 msgr2=0x7f2b240b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 0x7f2b240b6150 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009d60 tx=0x7f2b28009110 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 msgr2=0x7f2b3419e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00eb10 tx=0x7f2b2c00ef10 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 shutdown_connections 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b34108e70 0x7f2b3419e8c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b240b3c90 0x7f2b240b6150 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b341084f0 0x7f2b34199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 --2- 172.21.15.107:0/2854535851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b34102ae0 0x7f2b34198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:48.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 >> 172.21.15.107:0/2854535851 conn(0x7f2b340fe510 msgr2=0x7f2b34100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:48.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 shutdown_connections 2024-08-20T06:38:48.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:48.122+0000 7f2b3ac78640 1 -- 172.21.15.107:0/2854535851 wait complete. 2024-08-20T06:38:48.878 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:38:48.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:48 smithi107 ceph-mon[18495]: pgmap v386: 97 pgs: 97 active+clean; 75 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-08-20T06:38:48.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:48 smithi183 ceph-mon[22204]: pgmap v386: 97 pgs: 97 active+clean; 75 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-08-20T06:38:49.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:48 smithi193 ceph-mon[21944]: pgmap v386: 97 pgs: 97 active+clean; 75 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 132 op/s 2024-08-20T06:38:49.238 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:49.810 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 -- 172.21.15.107:0/2985086190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 msgr2=0x7f9cac10f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 --2- 172.21.15.107:0/2985086190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10f620 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4009ca0 tx=0x7f9ca402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 -- 172.21.15.107:0/2985086190 shutdown_connections 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 --2- 172.21.15.107:0/2985086190 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10f620 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 --2- 172.21.15.107:0/2985086190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac1084a0 0x7f9cac1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 --2- 172.21.15.107:0/2985086190 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac102a90 0x7f9cac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 -- 172.21.15.107:0/2985086190 >> 172.21.15.107:0/2985086190 conn(0x7f9cac0fe540 msgr2=0x7f9cac100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 -- 172.21.15.107:0/2985086190 shutdown_connections 2024-08-20T06:38:49.811 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.809+0000 7f9cb241a640 1 -- 172.21.15.107:0/2985086190 wait complete. 2024-08-20T06:38:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.810+0000 7f9cb241a640 1 Processor -- start 2024-08-20T06:38:49.813 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.810+0000 7f9cb241a640 1 -- start start 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac102a90 0x7f9cac109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1129a0 con 0x7f9cac108e20 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac112820 con 0x7f9cac1084a0 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb241a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac112b20 con 0x7f9cac102a90 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48140/0 (socket says 172.21.15.107:48140) 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 -- 172.21.15.107:0/135797485 learned_addr learned my addr 172.21.15.107:0/135797485 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 -- 172.21.15.107:0/135797485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac102a90 msgr2=0x7f9cac109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cabfff640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac102a90 0x7f9cac109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac102a90 0x7f9cac109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 -- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 msgr2=0x7f9cac10a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.811+0000 7f9cab7fe640 1 -- 172.21.15.107:0/135797485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac1ab660 con 0x7f9cac1084a0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000e9f0 tx=0x7f9ca000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9ca97fa640 1 -- 172.21.15.107:0/135797485 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca000cd80 con 0x7f9cac1084a0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9ca97fa640 1 -- 172.21.15.107:0/135797485 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca00054b0 con 0x7f9cac1084a0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac1ab950 con 0x7f9cac1084a0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.812+0000 7f9ca97fa640 1 -- 172.21.15.107:0/135797485 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca0010430 con 0x7f9cac1084a0 2024-08-20T06:38:49.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.813+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac1abe90 con 0x7f9cac1084a0 2024-08-20T06:38:49.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 Processor -- start 2024-08-20T06:38:49.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 -- start start 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fce2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9ca97fa640 1 -- 172.21.15.107:0/135797485 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f9ca00105f0 con 0x7f9cac1084a0 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f73fc59930 con 0x55f73fce2880 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f73fc48e30 con 0x55f73fcef130 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb4813740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f73fc4ce70 con 0x55f73fce31a0 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:54298/0 (socket says 172.21.15.107:54298) 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cabfff640 1 -- 172.21.15.107:0/2869991951 learned_addr learned my addr 172.21.15.107:0/2869991951 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:49.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fce2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.816+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 -- 172.21.15.107:0/2869991951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 msgr2=0x55f73fcf1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 -- 172.21.15.107:0/2869991951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 msgr2=0x55f73fce2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fce2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 -- 172.21.15.107:0/2869991951 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73f9a8650 con 0x55f73fce31a0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9cabfff640 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009c80 tx=0x7f9c9c010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8885cfcbf005cc8d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.817+0000 7f9c927fc640 1 -- 172.21.15.107:0/2869991951 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c004020 con 0x55f73fce31a0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.818+0000 7f9ca97fa640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 0x7f9c940b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.818+0000 7f9c927fc640 1 -- 172.21.15.107:0/2869991951 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9c9c0041c0 con 0x55f73fce31a0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.818+0000 7f9c927fc640 1 -- 172.21.15.107:0/2869991951 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c0126b0 con 0x55f73fce31a0 2024-08-20T06:38:49.820 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.818+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 0x7f9c940b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.818+0000 7f9ca97fa640 1 -- 172.21.15.107:0/135797485 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9ca00e40b0 con 0x7f9cac1084a0 2024-08-20T06:38:49.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- 172.21.15.107:0/2869991951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 msgr2=0x55f73fceebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 0x7f9c940b6110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4004af0 tx=0x7f9ca4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009c80 tx=0x7f9c9c010740 comp rx=0 tx=0).stop 2024-08-20T06:38:49.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- 172.21.15.107:0/2869991951 shutdown_connections 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fceebf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2869991951 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fce2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- 172.21.15.107:0/2869991951 >> 172.21.15.107:0/2869991951 conn(0x55f73fce06b0 msgr2=0x55f73fce1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- 172.21.15.107:0/2869991951 shutdown_connections 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- 172.21.15.107:0/2869991951 wait complete. 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.819+0000 7f9cb4813740 1 -- start start 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 Processor -- start 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f73faab700 con 0x55f73fce31a0 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f73fbcf810 con 0x55f73fce2880 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f73fcf1de0 con 0x55f73fcef130 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48156/0 (socket says 172.21.15.107:48156) 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cabfff640 1 -- 172.21.15.107:0/1214812275 learned_addr learned my addr 172.21.15.107:0/1214812275 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb4813740 10 client.0 _ll_register_callbacks cb 0x55f73fc42700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.820+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 -- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 msgr2=0x55f73fcecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 -- 172.21.15.107:0/1214812275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 msgr2=0x55f73fcec650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 -- 172.21.15.107:0/1214812275 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73fcfb210 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cabfff640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c003cb0 tx=0x7f9c9c009fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca42a22a1041be19 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c005440 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9c9c0055e0 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f73fbf8150 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c003df0 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cb4813740 10 client.25202 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.821+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f73fcfdba0 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f73fbf8150 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9c9c00f040 con 0x55f73fce2880 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 10 client.25202 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 10 client.25202 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:38:49.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 20 client.25202 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fcff000 0x55f73fd013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 4 client.25202 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f73fc5b880 con 0x55f73fcff000 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fcff000 0x55f73fd013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.822+0000 7f9c8b7fe640 10 client.25202 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:49.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fcff000 0x55f73fd013f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5dd426caaf676c8 server_cookie=e3f1aaeb8e55a686 in_seq=0 out_seq=0 2024-08-20T06:38:49.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/1214812275 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f73fc5b880 con 0x55f73fcff000 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9c8b7fe640 10 client.25202 handle_command_reply: tid=0 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9cb4813740 1 client.25202 shutdown 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 msgr2=0x55f73fcec110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.823+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c003cb0 tx=0x7f9c9c009fa0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 shutdown_connections 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fcff000 0x55f73fd013f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 --2- 172.21.15.107:0/1214812275 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fcec110 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9c8b7fe640 0 client.25202 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 >> 172.21.15.107:0/1214812275 conn(0x55f73fce06b0 msgr2=0x55f73fce6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 shutdown_connections 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 1 -- 172.21.15.107:0/1214812275 wait complete. 2024-08-20T06:38:49.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.824+0000 7f9cb4813740 20 client.25202 trim_cache size 0 max 16384 2024-08-20T06:38:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 Processor -- start 2024-08-20T06:38:49.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 -- start start 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fce6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b6eb0 con 0x55f73fce31a0 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b4780 con 0x55f73fce2880 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb4813740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b25b0 con 0x55f73fcef130 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fce6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fce6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48160/0 (socket says 172.21.15.107:48160) 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cabfff640 1 -- 172.21.15.107:0/2583373798 learned_addr learned my addr 172.21.15.107:0/2583373798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:49.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.842+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 -- 172.21.15.107:0/2583373798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 msgr2=0x55f73fcecb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 -- 172.21.15.107:0/2583373798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 msgr2=0x55f73fce6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fce6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 -- 172.21.15.107:0/2583373798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73fce04e0 con 0x55f73fce31a0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55f73fce2830 tx=0x7f9ca00e1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b871b2d6ff648188 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9c91ffb640 1 -- 172.21.15.107:0/2583373798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00bc070 con 0x55f73fce31a0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9c91ffb640 1 -- 172.21.15.107:0/2583373798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca00aaad0 con 0x55f73fce31a0 2024-08-20T06:38:49.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.843+0000 7f9c91ffb640 1 -- 172.21.15.107:0/2583373798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00b4070 con 0x55f73fce31a0 2024-08-20T06:38:49.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 -- 172.21.15.107:0/2583373798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 msgr2=0x55f73fcec610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55f73fce2830 tx=0x7f9ca00e1a40 comp rx=0 tx=0).stop 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 -- 172.21.15.107:0/2583373798 shutdown_connections 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fcef130 0x55f73fcecb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce31a0 0x55f73fcec610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 --2- 172.21.15.107:0/2583373798 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fce2880 0x55f73fce6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 -- 172.21.15.107:0/2583373798 >> 172.21.15.107:0/2583373798 conn(0x55f73fce06b0 msgr2=0x55f73fcedf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 -- 172.21.15.107:0/2583373798 shutdown_connections 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.844+0000 7f9cb4813740 1 -- 172.21.15.107:0/2583373798 wait complete. 2024-08-20T06:38:49.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 -- start start 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 Processor -- start 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fcf2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b2c40 con 0x55f73fce2880 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b40b0 con 0x55f73fcef130 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac1b3310 con 0x55f73fce31a0 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb4813740 10 client.0 _ll_register_callbacks cb 0x7f9ca00e79c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb09b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb09b7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48172/0 (socket says 172.21.15.107:48172) 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cb09b7640 1 -- 172.21.15.107:0/3026925822 learned_addr learned my addr 172.21.15.107:0/3026925822 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:49.852 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cabfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46804/0 (socket says 172.21.15.107:46804) 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.845+0000 7f9cab7fe640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fcf2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 -- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 msgr2=0x55f73fcf2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fcf2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 -- 172.21.15.107:0/3026925822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 msgr2=0x55f73fcf7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 -- 172.21.15.107:0/3026925822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73fd01950 con 0x55f73fcef130 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cabfff640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb09b7640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4031920 tx=0x7f9ca403e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad63b63a2552a57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9c8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4055060 con 0x55f73fcef130 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9ca4003d80 con 0x55f73fcef130 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ca00aaad0 con 0x55f73fcef130 2024-08-20T06:38:49.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ca4004050 con 0x55f73fcef130 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.846+0000 7f9cb4813740 10 client.25208 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f73fd69850 con 0x55f73fcef130 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9ca00aaad0 con 0x55f73fcef130 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9ca4043040 con 0x55f73fcef130 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 10 client.25208 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 10 client.25208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 20 client.25208 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fd6a030 0x55f73fd6c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 4 client.25208 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f73fc57320 con 0x55f73fd6a030 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.847+0000 7f9cabfff640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fd6a030 0x55f73fd6c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:49.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9c8b7fe640 10 client.25208 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cabfff640 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fd6a030 0x55f73fd6c450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1391d6c7494d06f server_cookie=d53729c9dc57dd96 in_seq=0 out_seq=0 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9c8b7fe640 1 -- 172.21.15.107:0/3026925822 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f73fc57320 con 0x55f73fd6a030 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9c8b7fe640 10 client.25208 handle_command_reply: tid=0 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 client.25208 shutdown 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 msgr2=0x55f73fcf2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4031920 tx=0x7f9ca403e040 comp rx=0 tx=0).stop 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 shutdown_connections 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55f73fd6a030 0x55f73fd6c450 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f73fcef130 0x55f73fcf2f70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f73fce31a0 0x55f73fcf2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 --2- 172.21.15.107:0/3026925822 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f73fce2880 0x55f73fcf7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9c8b7fe640 0 client.25208 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 >> 172.21.15.107:0/3026925822 conn(0x55f73fce06b0 msgr2=0x55f73fcdef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.848+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 shutdown_connections 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb4813740 1 -- 172.21.15.107:0/3026925822 wait complete. 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb4813740 20 client.25208 trim_cache size 0 max 16384 2024-08-20T06:38:49.856 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 msgr2=0x7f9c940b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 0x7f9c940b6110 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4004af0 tx=0x7f9ca4005d80 comp rx=0 tx=0).stop 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 msgr2=0x7f9cac109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.849+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ca000e9f0 tx=0x7f9ca000edf0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 shutdown_connections 2024-08-20T06:38:49.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cac108e20 0x7f9cac10a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9cac1084a0 0x7f9cac109b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9c940b3c50 0x7f9c940b6110 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 --2- 172.21.15.107:0/135797485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cac102a90 0x7f9cac109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 >> 172.21.15.107:0/135797485 conn(0x7f9cac0fe540 msgr2=0x7f9cac106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 shutdown_connections 2024-08-20T06:38:49.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:49.850+0000 7f9cb241a640 1 -- 172.21.15.107:0/135797485 wait complete. 2024-08-20T06:38:50.713 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:38:50.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:50 smithi183 ceph-mon[22204]: pgmap v387: 97 pgs: 97 active+clean; 76 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:51.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:50 smithi193 ceph-mon[21944]: pgmap v387: 97 pgs: 97 active+clean; 76 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:51.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:50 smithi107 ceph-mon[18495]: pgmap v387: 97 pgs: 97 active+clean; 76 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:51.125 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:52.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:52 smithi107 ceph-mon[18495]: pgmap v388: 97 pgs: 97 active+clean; 81 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:52.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:52 smithi183 ceph-mon[22204]: pgmap v388: 97 pgs: 97 active+clean; 81 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:53.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:52 smithi193 ceph-mon[21944]: pgmap v388: 97 pgs: 97 active+clean; 81 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 2.0 MiB/s wr, 125 op/s 2024-08-20T06:38:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.130+0000 7fe56fb14640 1 -- 172.21.15.107:0/1007111784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 msgr2=0x7fe568111530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.130+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1007111784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe568111530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe560009be0 tx=0x7fe56002a750 comp rx=0 tx=0).stop 2024-08-20T06:38:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 -- 172.21.15.107:0/1007111784 shutdown_connections 2024-08-20T06:38:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1007111784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe568111530 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1007111784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56810ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1007111784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe568101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 -- 172.21.15.107:0/1007111784 >> 172.21.15.107:0/1007111784 conn(0x7fe5680fd400 msgr2=0x7fe5680ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 -- 172.21.15.107:0/1007111784 shutdown_connections 2024-08-20T06:38:54.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 -- 172.21.15.107:0/1007111784 wait complete. 2024-08-20T06:38:54.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.131+0000 7fe56fb14640 1 Processor -- start 2024-08-20T06:38:54.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 -- start start 2024-08-20T06:38:54.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56819b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe5681a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe568114b30 con 0x7fe568101590 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681149b0 con 0x7fe568101eb0 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56fb14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe568114cb0 con 0x7fe56810f140 2024-08-20T06:38:54.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46818/0 (socket says 172.21.15.107:46818) 2024-08-20T06:38:54.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56d8b0640 1 -- 172.21.15.107:0/1651240440 learned_addr learned my addr 172.21.15.107:0/1651240440 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:54.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56819b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.132+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe5681a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 -- 172.21.15.107:0/1651240440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 msgr2=0x7fe5681a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe5681a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 -- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 msgr2=0x7fe56819b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 -- 172.21.15.107:0/1651240440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5681a1190 con 0x7fe568101eb0 2024-08-20T06:38:54.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56819b620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e9f0 tx=0x7fe55800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe55effd640 1 -- 172.21.15.107:0/1651240440 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55800ccc0 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5681a1480 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe55effd640 1 -- 172.21.15.107:0/1651240440 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe5580108c0 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe55effd640 1 -- 172.21.15.107:0/1651240440 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580187e0 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.133+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5681a7d10 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.136+0000 7fe55effd640 1 -- 172.21.15.107:0/1651240440 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fe558010430 con 0x7fe568101eb0 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 Processor -- start 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 -- start start 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a090 0x555e6223a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62248d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e621a8930 con 0x555e6223a090 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e62197e30 con 0x555e62246940 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe571f0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e6219be70 con 0x555e6223a9b0 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe55effd640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 0x7fe5540b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.137+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:54356/0 (socket says 172.21.15.107:54356) 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 -- 172.21.15.107:0/3469031820 learned_addr learned my addr 172.21.15.107:0/3469031820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d0af640 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62248d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a090 0x555e6223a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 0x7fe5540b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 -- 172.21.15.107:0/3469031820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 msgr2=0x555e62248d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62248d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 -- 172.21.15.107:0/3469031820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a090 msgr2=0x555e6223a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a090 0x555e6223a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 -- 172.21.15.107:0/3469031820 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e61ef7650 con 0x555e6223a9b0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe55effd640 1 -- 172.21.15.107:0/1651240440 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe5580e3dc0 con 0x7fe568101eb0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5640115c0 tx=0x7fe564011a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a1bba70fca6ee7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.138+0000 7fe553fff640 1 -- 172.21.15.107:0/3469031820 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56400f6c0 con 0x555e6223a9b0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.139+0000 7fe553fff640 1 -- 172.21.15.107:0/3469031820 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe56400fcf0 con 0x555e6223a9b0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.139+0000 7fe553fff640 1 -- 172.21.15.107:0/3469031820 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe564019d70 con 0x555e6223a9b0 2024-08-20T06:38:54.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.139+0000 7fe571f0d740 1 -- 172.21.15.107:0/3469031820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 msgr2=0x555e62246400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.139+0000 7fe571f0d740 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5640115c0 tx=0x7fe564011a80 comp rx=0 tx=0).stop 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.139+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 0x7fe5540b6110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe56002ac70 tx=0x7fe56002a750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 -- 172.21.15.107:0/3469031820 shutdown_connections 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62248d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a9b0 0x555e62246400 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 --2- 172.21.15.107:0/3469031820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a090 0x555e6223a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 -- 172.21.15.107:0/3469031820 >> 172.21.15.107:0/3469031820 conn(0x555e62237ec0 msgr2=0x555e62239060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 -- 172.21.15.107:0/3469031820 shutdown_connections 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 -- 172.21.15.107:0/3469031820 wait complete. 2024-08-20T06:38:54.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 -- start start 2024-08-20T06:38:54.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.140+0000 7fe571f0d740 1 Processor -- start 2024-08-20T06:38:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62246310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62243e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555e61ffa700 con 0x555e6223a9b0 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46850/0 (socket says 172.21.15.107:46850) 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 -- 172.21.15.107:0/1598294707 learned_addr learned my addr 172.21.15.107:0/1598294707 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56e0b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62243e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555e6211e810 con 0x555e62246940 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555e622495c0 con 0x555e6223a090 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe571f0d740 10 client.0 _ll_register_callbacks cb 0x555e62191700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62246310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 -- 172.21.15.107:0/1598294707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 msgr2=0x555e62246310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62246310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 -- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 msgr2=0x555e62243e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62243e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.141+0000 7fe56d0af640 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e62252950 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62243e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62246310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe55800cfd0 tx=0x7fe5580aa5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af352addda50bf15 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe5517fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5580aadc0 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe558010a70 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555e62147150 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5580af310 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe571f0d740 10 client.15711 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.142+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555e62255250 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x555e62147150 con 0x555e6223a9b0 2024-08-20T06:38:54.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe5580a6890 con 0x555e6223a9b0 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 10 client.15711 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 20 client.15711 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e62256710 0x555e62258b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 4 client.15711 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.143+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e621aa880 con 0x555e62256710 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.144+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e62256710 0x555e62258b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.144+0000 7fe5517fa640 10 client.15711 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.144+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e62256710 0x555e62258b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e7c0dd8aafd632 server_cookie=af1791812c47d314 in_seq=0 out_seq=0 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe5517fa640 1 -- 172.21.15.107:0/1598294707 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e621aa880 con 0x555e62256710 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe5517fa640 10 client.15711 handle_command_reply: tid=0 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 client.15711 shutdown 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 msgr2=0x555e62243760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe55800cfd0 tx=0x7fe5580aa5f0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 shutdown_connections 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e62243e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62243760 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e62256710 0x555e62258b20 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1598294707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62246310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 >> 172.21.15.107:0/1598294707 conn(0x555e62237ec0 msgr2=0x555e6223da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.145+0000 7fe5517fa640 0 client.15711 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.146+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 shutdown_connections 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.146+0000 7fe571f0d740 1 -- 172.21.15.107:0/1598294707 wait complete. 2024-08-20T06:38:54.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.146+0000 7fe571f0d740 20 client.15711 trim_cache size 0 max 16384 2024-08-20T06:38:54.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 Processor -- start 2024-08-20T06:38:54.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 -- start start 2024-08-20T06:38:54.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62258b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e6225e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681ae8e0 con 0x555e6223a9b0 2024-08-20T06:38:54.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681ac1b0 con 0x555e62246940 2024-08-20T06:38:54.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe571f0d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681aa000 con 0x555e6223a090 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe56e0b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe56e0b1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48200/0 (socket says 172.21.15.107:48200) 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe56e0b1640 1 -- 172.21.15.107:0/525925908 learned_addr learned my addr 172.21.15.107:0/525925908 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.165+0000 7fe56d0af640 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e6225e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 -- 172.21.15.107:0/525925908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 msgr2=0x555e62258b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T06:38:54.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62258b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 -- 172.21.15.107:0/525925908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 msgr2=0x555e6225e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e6225e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 -- 172.21.15.107:0/525925908 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e62238ba0 con 0x555e62246940 2024-08-20T06:38:54.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe560033000 tx=0x7fe560041000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fed02cb02f153ba server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe5537fe640 1 -- 172.21.15.107:0/525925908 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560051070 con 0x555e62246940 2024-08-20T06:38:54.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe5537fe640 1 -- 172.21.15.107:0/525925908 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe560037040 con 0x555e62246940 2024-08-20T06:38:54.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.166+0000 7fe5537fe640 1 -- 172.21.15.107:0/525925908 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560036040 con 0x555e62246940 2024-08-20T06:38:54.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 -- 172.21.15.107:0/525925908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 msgr2=0x555e6225e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe560033000 tx=0x7fe560041000 comp rx=0 tx=0).stop 2024-08-20T06:38:54.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 -- 172.21.15.107:0/525925908 shutdown_connections 2024-08-20T06:38:54.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e62246940 0x555e6225e980 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e6225e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 --2- 172.21.15.107:0/525925908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e6223a090 0x555e62258b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 -- 172.21.15.107:0/525925908 >> 172.21.15.107:0/525925908 conn(0x555e62237ec0 msgr2=0x555e6223e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 -- 172.21.15.107:0/525925908 shutdown_connections 2024-08-20T06:38:54.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.167+0000 7fe571f0d740 1 -- 172.21.15.107:0/525925908 wait complete. 2024-08-20T06:38:54.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.168+0000 7fe571f0d740 1 -- start start 2024-08-20T06:38:54.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.168+0000 7fe571f0d740 1 Processor -- start 2024-08-20T06:38:54.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe56d8b0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:48210/0 (socket says 172.21.15.107:48210) 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe56d0af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe56d0af640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46880/0 (socket says 172.21.15.107:46880) 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe56d8b0640 1 -- 172.21.15.107:0/1352711439 learned_addr learned my addr 172.21.15.107:0/1352711439 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:54.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e62246940 0x555e6224d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681aa670 con 0x555e6223a9b0 2024-08-20T06:38:54.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681abae0 con 0x555e6223a090 2024-08-20T06:38:54.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe5681aad40 con 0x555e62246940 2024-08-20T06:38:54.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.169+0000 7fe571f0d740 10 client.0 _ll_register_callbacks cb 0x7fe5580aa710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:54.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56e0b1640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e62246940 0x555e6224d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 -- 172.21.15.107:0/1352711439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e62246940 msgr2=0x555e6224d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e62246940 0x555e6224d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 -- 172.21.15.107:0/1352711439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 msgr2=0x555e62247690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e6224ddf0 con 0x555e6223a090 2024-08-20T06:38:54.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560037040 con 0x555e6223a090 2024-08-20T06:38:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe56d8b0640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe564005c80 tx=0x7fe564009fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486337e6b24a6cfa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe5517fa640 10 client.25226 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.170+0000 7fe5517fa640 1 -- 172.21.15.107:0/1352711439 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640050b0 con 0x555e6223a090 2024-08-20T06:38:54.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe5517fa640 1 -- 172.21.15.107:0/1352711439 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe564005250 con 0x555e6223a090 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe5517fa640 1 -- 172.21.15.107:0/1352711439 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe5640054b0 con 0x555e6223a090 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 10 client.25226 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=25}) -- 0x555e6224eff0 con 0x555e6223a090 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe5517fa640 1 -- 172.21.15.107:0/1352711439 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fe56400f6c0 con 0x555e6223a090 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 10 client.25226 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:54.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 10 client.25226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:54.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 20 client.25226 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:54.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e622c1410 0x555e622c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:54.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 4 client.25226 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:38:54.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555e6222a880 con 0x555e622c1410 2024-08-20T06:38:54.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.171+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e622c1410 0x555e622c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe56d0af640 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e622c1410 0x555e622c3800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda4ee96d7c7a014 server_cookie=cebb7fde34a4f4b7 in_seq=0 out_seq=0 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe5517fa640 10 client.25226 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe5517fa640 1 -- 172.21.15.107:0/1352711439 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555e6222a880 con 0x555e622c1410 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe5517fa640 10 client.25226 handle_command_reply: tid=0 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe571f0d740 1 client.25226 shutdown 2024-08-20T06:38:54.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 msgr2=0x555e62247150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.172+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe564005c80 tx=0x7fe564009fc0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 shutdown_connections 2024-08-20T06:38:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555e622c1410 0x555e622c3800 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e62246940 0x555e6224d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555e6223a9b0 0x555e62247690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 --2- 172.21.15.107:0/1352711439 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555e6223a090 0x555e62247150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 >> 172.21.15.107:0/1352711439 conn(0x555e62237ec0 msgr2=0x555e62238dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe5517fa640 0 client.25226 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 shutdown_connections 2024-08-20T06:38:54.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 1 -- 172.21.15.107:0/1352711439 wait complete. 2024-08-20T06:38:54.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.173+0000 7fe571f0d740 20 client.25226 trim_cache size 0 max 16384 2024-08-20T06:38:54.192 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:38:54.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 msgr2=0x7fe5540b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 0x7fe5540b6110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe56002ac70 tx=0x7fe56002a750 comp rx=0 tx=0).stop 2024-08-20T06:38:54.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 msgr2=0x7fe56819b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:54.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56819b620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e9f0 tx=0x7fe55800eeb0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 shutdown_connections 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe5540b3c50 0x7fe5540b6110 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe56810f140 0x7fe5681a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe568101eb0 0x7fe56819b620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 --2- 172.21.15.107:0/1651240440 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe568101590 0x7fe56819b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 >> 172.21.15.107:0/1651240440 conn(0x7fe5680fd400 msgr2=0x7fe5680ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:54.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 shutdown_connections 2024-08-20T06:38:54.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:54.174+0000 7fe56fb14640 1 -- 172.21.15.107:0/1651240440 wait complete. 2024-08-20T06:38:54.827 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:54 smithi107 ceph-mon[18495]: pgmap v389: 97 pgs: 97 active+clean; 85 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-08-20T06:38:54.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:54 smithi183 ceph-mon[22204]: pgmap v389: 97 pgs: 97 active+clean; 85 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-08-20T06:38:55.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:54 smithi193 ceph-mon[21944]: pgmap v389: 97 pgs: 97 active+clean; 85 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-08-20T06:38:55.885 INFO:tasks.workunit.client.0.smithi107.stdout: 19 14094 257 9820 199 4712 150 2024-08-20T06:38:55.923 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:38:55.924 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:38:56.293 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 -- 172.21.15.107:0/2167514961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d8104cb0 msgr2=0x7f45d8105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/2167514961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d8104cb0 0x7f45d8105090 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009900 tx=0x7f45c802a410 comp rx=0 tx=0).stop 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 -- 172.21.15.107:0/2167514961 shutdown_connections 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/2167514961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 0x7f45d8111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/2167514961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d810a6c0 0x7f45d810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/2167514961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d8104cb0 0x7f45d8105090 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 -- 172.21.15.107:0/2167514961 >> 172.21.15.107:0/2167514961 conn(0x7f45d8100720 msgr2=0x7f45d8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 -- 172.21.15.107:0/2167514961 shutdown_connections 2024-08-20T06:38:56.842 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.840+0000 7f45dd1dd640 1 -- 172.21.15.107:0/2167514961 wait complete. 2024-08-20T06:38:56.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.841+0000 7f45dd1dd640 1 Processor -- start 2024-08-20T06:38:56.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.841+0000 7f45dd1dd640 1 -- start start 2024-08-20T06:38:56.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d810a6c0 0x7f45d819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 0x7f45d81a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.844 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8114bc0 con 0x7f45d810a6c0 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8114a40 con 0x7f45d8104cb0 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45dd1dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d8114d40 con 0x7f45d810b040 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d810a6c0 0x7f45d819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:54664/0 (socket says 172.21.15.107:54664) 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 -- 172.21.15.107:0/1960452125 learned_addr learned my addr 172.21.15.107:0/1960452125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d759e640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 0x7f45d81a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 -- 172.21.15.107:0/1960452125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 msgr2=0x7f45d81a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 0x7f45d81a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 -- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d810a6c0 msgr2=0x7f45d819b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d810a6c0 0x7f45d819b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.845 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.842+0000 7f45d6d9d640 1 -- 172.21.15.107:0/1960452125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d81a1160 con 0x7f45d8104cb0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f45c802a930 tx=0x7f45c8003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45bffff640 1 -- 172.21.15.107:0/1960452125 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c803f070 con 0x7f45d8104cb0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45bffff640 1 -- 172.21.15.107:0/1960452125 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45c803baf0 con 0x7f45d8104cb0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d81a13f0 con 0x7f45d8104cb0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45bffff640 1 -- 172.21.15.107:0/1960452125 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8035030 con 0x7f45d8104cb0 2024-08-20T06:38:56.846 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.843+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d81a7b10 con 0x7f45d8104cb0 2024-08-20T06:38:56.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.846+0000 7f45bffff640 1 -- 172.21.15.107:0/1960452125 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f45c8042070 con 0x7f45d8104cb0 2024-08-20T06:38:56.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 Processor -- start 2024-08-20T06:38:56.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 -- start start 2024-08-20T06:38:56.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5950720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea5953050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ea58ba940 con 0x562ea5950c60 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562ea58a9e40 con 0x562ea5944cd0 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45df5d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ea58ade80 con 0x562ea59443b0 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45bffff640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 0x7f45c00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5950720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45d759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.847+0000 7f45d759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:39954/0 (socket says 172.21.15.107:39954) 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 -- 172.21.15.107:0/3192439300 learned_addr learned my addr 172.21.15.107:0/3192439300 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45bffff640 1 -- 172.21.15.107:0/1960452125 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f45c8104160 con 0x7f45d8104cb0 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d659c640 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea5953050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 0x7f45c00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 -- 172.21.15.107:0/3192439300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 msgr2=0x562ea5950720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5950720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 -- 172.21.15.107:0/3192439300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 msgr2=0x562ea5953050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea5953050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d759e640 1 -- 172.21.15.107:0/3192439300 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea5846520 con 0x562ea59443b0 2024-08-20T06:38:56.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.848+0000 7f45d659c640 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea5953050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.849+0000 7f45d759e640 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45d0013b20 tx=0x7f45d000f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6296e0f134ba0609 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.849+0000 7f45d759e640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 0x7f45c00b6110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f45d000f8c0 tx=0x7f45d001f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.849+0000 7f45bd7fa640 1 -- 172.21.15.107:0/3192439300 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0003e00 con 0x562ea59443b0 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.849+0000 7f45bd7fa640 1 -- 172.21.15.107:0/3192439300 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45d0022020 con 0x562ea59443b0 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.849+0000 7f45bd7fa640 1 -- 172.21.15.107:0/3192439300 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0010690 con 0x562ea59443b0 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- 172.21.15.107:0/3192439300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 msgr2=0x562ea5944790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45d0013b20 tx=0x7f45d000f3c0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- 172.21.15.107:0/3192439300 shutdown_connections 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea5953050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5950720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 --2- 172.21.15.107:0/3192439300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea5944790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- 172.21.15.107:0/3192439300 >> 172.21.15.107:0/3192439300 conn(0x562ea59421e0 msgr2=0x562ea5943380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- 172.21.15.107:0/3192439300 shutdown_connections 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- 172.21.15.107:0/3192439300 wait complete. 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 -- start start 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.850+0000 7f45df5d6740 1 Processor -- start 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea594dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea594e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5950c60 0x562ea594e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x562ea570c890 con 0x562ea59443b0 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x562ea5830700 con 0x562ea5944cd0 2024-08-20T06:38:56.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x562ea5953950 con 0x562ea5950c60 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45df5d6740 10 client.0 _ll_register_callbacks cb 0x562ea58a56f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5950c60 0x562ea594e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea594dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea594dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56908/0 (socket says 172.21.15.107:56908) 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d6d9d640 1 -- 172.21.15.107:0/1187403404 learned_addr learned my addr 172.21.15.107:0/1187403404 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea594e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 -- 172.21.15.107:0/1187403404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5950c60 msgr2=0x562ea594e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5950c60 0x562ea594e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 -- 172.21.15.107:0/1187403404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 msgr2=0x562ea594dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea594dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.851+0000 7f45d659c640 1 -- 172.21.15.107:0/1187403404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea5609930 con 0x562ea5944cd0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45d659c640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea594e1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45d810a160 tx=0x7f45cc012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b51d5b5e63cc16 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45baff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004030 con 0x562ea5944cd0 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f45cc0041d0 con 0x562ea5944cd0 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d0022020 con 0x562ea5944cd0 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45cc0043d0 con 0x562ea5944cd0 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45df5d6740 10 client.25238 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:56.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ea595f5c0 con 0x562ea5944cd0 2024-08-20T06:38:56.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.852+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45d0022020 con 0x562ea5944cd0 2024-08-20T06:38:56.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45cc003a20 con 0x562ea5944cd0 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 10 client.25238 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 20 client.25238 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea5960ad0 0x562ea5962ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 4 client.25238 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ea58bc890 con 0x562ea5960ad0 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea5960ad0 0x562ea5962ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45baff5640 10 client.25238 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.853+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea5960ad0 0x562ea5962ef0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e724e918b413627 server_cookie=1e563290e0cb31af in_seq=0 out_seq=0 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45baff5640 1 -- 172.21.15.107:0/1187403404 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ea58bc890 con 0x562ea5960ad0 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45baff5640 10 client.25238 handle_command_reply: tid=0 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 client.25238 shutdown 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 msgr2=0x562ea594e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea594e1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45d810a160 tx=0x7f45cc012b60 comp rx=0 tx=0).stop 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 shutdown_connections 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea5960ad0 0x562ea5962ef0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45baff5640 0 client.25238 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:56.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5950c60 0x562ea594e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea594e1c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1187403404 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea594dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.854+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 >> 172.21.15.107:0/1187403404 conn(0x562ea59421e0 msgr2=0x562ea5947d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.855+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 shutdown_connections 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.855+0000 7f45df5d6740 1 -- 172.21.15.107:0/1187403404 wait complete. 2024-08-20T06:38:56.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.855+0000 7f45df5d6740 20 client.25238 trim_cache size 0 max 16384 2024-08-20T06:38:56.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 Processor -- start 2024-08-20T06:38:56.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 -- start start 2024-08-20T06:38:56.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea5941300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 0x562ea5941840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b2aa0 con 0x562ea59443b0 2024-08-20T06:38:56.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81b0370 con 0x562ea5950c60 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45df5d6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81ae1a0 con 0x562ea5944cd0 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45d759e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45d759e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:54690/0 (socket says 172.21.15.107:54690) 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45d6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea5941300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.877+0000 7f45d759e640 1 -- 172.21.15.107:0/2407903143 learned_addr learned my addr 172.21.15.107:0/2407903143 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d659c640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 0x562ea5941840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 -- 172.21.15.107:0/2407903143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 msgr2=0x562ea5941840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 0x562ea5941840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 -- 172.21.15.107:0/2407903143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 msgr2=0x562ea5941300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea5941300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 -- 172.21.15.107:0/2407903143 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ea5942c60 con 0x562ea5950c60 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea5941300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:56.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d659c640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 0x562ea5941840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:56.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45d759e640 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45d0013b20 tx=0x7f45d0005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e4c22df781193d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45bcff9640 1 -- 172.21.15.107:0/2407903143 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d00136c0 con 0x562ea5950c60 2024-08-20T06:38:56.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45bcff9640 1 -- 172.21.15.107:0/2407903143 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45d0022040 con 0x562ea5950c60 2024-08-20T06:38:56.881 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.878+0000 7f45bcff9640 1 -- 172.21.15.107:0/2407903143 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004df0 con 0x562ea5950c60 2024-08-20T06:38:56.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 -- 172.21.15.107:0/2407903143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 msgr2=0x562ea59687a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.883 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45d0013b20 tx=0x7f45d0005f50 comp rx=0 tx=0).stop 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 -- 172.21.15.107:0/2407903143 shutdown_connections 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5950c60 0x562ea59687a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea5944cd0 0x562ea5941840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 --2- 172.21.15.107:0/2407903143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea59443b0 0x562ea5941300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.879+0000 7f45df5d6740 1 -- 172.21.15.107:0/2407903143 >> 172.21.15.107:0/2407903143 conn(0x562ea59421e0 msgr2=0x562ea5947ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.880+0000 7f45df5d6740 1 -- 172.21.15.107:0/2407903143 shutdown_connections 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.880+0000 7f45df5d6740 1 -- 172.21.15.107:0/2407903143 wait complete. 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.880+0000 7f45df5d6740 1 -- start start 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.880+0000 7f45df5d6740 1 Processor -- start 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea59555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5955b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea59cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.884 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81ae830 con 0x562ea5950c60 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81afca0 con 0x562ea5944cd0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45d659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5955b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45d659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5955b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:54704/0 (socket says 172.21.15.107:54704) 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45d659c640 1 -- 172.21.15.107:0/1866785251 learned_addr learned my addr 172.21.15.107:0/1866785251 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea59555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45d759e640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea59cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f45d81aef00 con 0x562ea59443b0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.881+0000 7f45df5d6740 10 client.0 _ll_register_callbacks cb 0x7f45cc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 -- 172.21.15.107:0/1866785251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 msgr2=0x562ea59555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea59555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 -- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 msgr2=0x562ea5955b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5955b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.885 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 -- 172.21.15.107:0/1866785251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ea59cd7b0 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45d759e640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea59cd0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a010 tx=0x7f45d0009e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfb7c2fe7402ce2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45baff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004b70 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f45d0004d10 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45d0022040 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.882+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45d0004410 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 10 client.15735 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562ea59d4c60 con 0x562ea5950c60 2024-08-20T06:38:56.886 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f45d0022040 con 0x562ea5950c60 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f45d003dad0 con 0x562ea5950c60 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 10 client.15735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 20 client.15735 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea59d51b0 0x562ea59d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 4 client.15735 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.883+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562ea58b8330 con 0x562ea59d51b0 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.884+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea59d51b0 0x562ea59d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.884+0000 7f45d6d9d640 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea59d51b0 0x562ea59d75d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddabefd50f9bdb6 server_cookie=29eec3bccd2eb84c in_seq=0 out_seq=0 2024-08-20T06:38:56.888 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.884+0000 7f45baff5640 10 client.15735 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45baff5640 1 -- 172.21.15.107:0/1866785251 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562ea58b8330 con 0x562ea59d51b0 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45baff5640 10 client.15735 handle_command_reply: tid=0 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 client.15735 shutdown 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 msgr2=0x562ea59cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea59cd0a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f45d000a010 tx=0x7f45d0009e10 comp rx=0 tx=0).stop 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 shutdown_connections 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x562ea5950c60 0x562ea59cd0a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x562ea5944cd0 0x562ea5955b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x562ea59d51b0 0x562ea59d75d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45baff5640 0 client.15735 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 --2- 172.21.15.107:0/1866785251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562ea59443b0 0x562ea59555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 >> 172.21.15.107:0/1866785251 conn(0x562ea59421e0 msgr2=0x562ea5941300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.889 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 shutdown_connections 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 1 -- 172.21.15.107:0/1866785251 wait complete. 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.885+0000 7f45df5d6740 20 client.15735 trim_cache size 0 max 16384 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "c1408381-e30c-42d7-83ed-686af9a0a0e2", 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:38:56.890 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 msgr2=0x7f45c00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 0x7f45c00b6110 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f45d000f8c0 tx=0x7f45d001f000 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 msgr2=0x7f45d819b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f45c802a930 tx=0x7f45c8003b90 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 shutdown_connections 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.886+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f45c00b3c50 0x7f45c00b6110 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45d810b040 0x7f45d81a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45d810a6c0 0x7f45d819b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 --2- 172.21.15.107:0/1960452125 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f45d8104cb0 0x7f45d819b0b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 >> 172.21.15.107:0/1960452125 conn(0x7f45d8100720 msgr2=0x7f45d8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 shutdown_connections 2024-08-20T06:38:56.892 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:56.887+0000 7f45dd1dd640 1 -- 172.21.15.107:0/1960452125 wait complete. 2024-08-20T06:38:56.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:56 smithi183 ceph-mon[22204]: pgmap v390: 97 pgs: 97 active+clean; 86 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-20T06:38:57.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:56 smithi193 ceph-mon[21944]: pgmap v390: 97 pgs: 97 active+clean; 86 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-20T06:38:57.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:56 smithi107 ceph-mon[18495]: pgmap v390: 97 pgs: 97 active+clean; 86 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 1.7 MiB/s wr, 102 op/s 2024-08-20T06:38:57.617 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:38:57.977 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:38:57.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:57 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:38:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:57 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:57 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:38:57.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:57 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:58.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:57 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:38:58.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:57 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:58.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:57 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:38:58.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:57 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:58.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:57 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:38:58.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:57 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:58.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:57 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:38:58.104 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:57 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:38:58.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 -- 172.21.15.107:0/3919234953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 msgr2=0x7f4e601077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 --2- 172.21.15.107:0/3919234953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601077e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009860 tx=0x7f4e5002a340 comp rx=0 tx=0).stop 2024-08-20T06:38:58.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 -- 172.21.15.107:0/3919234953 shutdown_connections 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 --2- 172.21.15.107:0/3919234953 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60107d20 0x7f4e6010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 --2- 172.21.15.107:0/3919234953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601077e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 --2- 172.21.15.107:0/3919234953 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60101990 0x7f4e60101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 -- 172.21.15.107:0/3919234953 >> 172.21.15.107:0/3919234953 conn(0x7f4e600fd400 msgr2=0x7f4e600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 -- 172.21.15.107:0/3919234953 shutdown_connections 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.494+0000 7f4e672ee640 1 -- 172.21.15.107:0/3919234953 wait complete. 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 Processor -- start 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 -- start start 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 0x7f4e6019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e6508a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e6508a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:54726/0 (socket says 172.21.15.107:54726) 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e64889640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e6508a640 1 -- 172.21.15.107:0/2379288112 learned_addr learned my addr 172.21.15.107:0/2379288112 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60111820 con 0x7f4e60107d20 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601116a0 con 0x7f4e60101990 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601119a0 con 0x7f4e601073a0 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.495+0000 7f4e6588b640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 0x7f4e6019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 -- 172.21.15.107:0/2379288112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 msgr2=0x7f4e601980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 -- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 msgr2=0x7f4e6019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 0x7f4e6019d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6019dc40 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6588b640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 0x7f4e6019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e6508a640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9c0 tx=0x7f4e5c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e567fc640 1 -- 172.21.15.107:0/2379288112 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00cde0 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e567fc640 1 -- 172.21.15.107:0/2379288112 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e5c0035f0 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e567fc640 1 -- 172.21.15.107:0/2379288112 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c010690 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6019df30 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.496+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6006a7b0 con 0x7f4e60101990 2024-08-20T06:38:58.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 Processor -- start 2024-08-20T06:38:58.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 -- start start 2024-08-20T06:38:58.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e567fc640 1 -- 172.21.15.107:0/2379288112 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f4e5c004000 con 0x7f4e60101990 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 0x561cce0a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce0a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561cce00b8e0 con 0x561cce095150 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ccdffade0 con 0x561cce094830 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.500+0000 7f4e696e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561ccdffee20 con 0x561cce0a10e0 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:54734/0 (socket says 172.21.15.107:54734) 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 -- 172.21.15.107:0/44409257 learned_addr learned my addr 172.21.15.107:0/44409257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e64889640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce0a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6508a640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 0x561cce0a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 -- 172.21.15.107:0/44409257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 msgr2=0x561cce0a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce0a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 -- 172.21.15.107:0/44409257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 msgr2=0x561cce0a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 0x561cce0a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 -- 172.21.15.107:0/44409257 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccdd5a650 con 0x561cce094830 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e567fc640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 0x7f4e4c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.501+0000 7f4e6588b640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009f60 tx=0x7f4e58009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cbcf2dcfee5aa03 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e567fc640 1 -- 172.21.15.107:0/2379288112 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4e5c0e3f80 con 0x7f4e60101990 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e6508a640 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 0x561cce0a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:38:58.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e4b7fe640 1 -- 172.21.15.107:0/44409257 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58011640 con 0x561cce094830 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e4b7fe640 1 -- 172.21.15.107:0/44409257 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e58011c70 con 0x561cce094830 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e4b7fe640 1 -- 172.21.15.107:0/44409257 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58010c90 con 0x561cce094830 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.502+0000 7f4e6508a640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 0x7f4e4c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- 172.21.15.107:0/44409257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 msgr2=0x561cce094c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009f60 tx=0x7f4e58009f90 comp rx=0 tx=0).stop 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e6508a640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 0x7f4e4c0b6090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0a8140 tx=0x7f4e5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- 172.21.15.107:0/44409257 shutdown_connections 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce0a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce095150 0x561cce0a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 --2- 172.21.15.107:0/44409257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce094c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- 172.21.15.107:0/44409257 >> 172.21.15.107:0/44409257 conn(0x561cce092660 msgr2=0x561cce093800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- 172.21.15.107:0/44409257 shutdown_connections 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- 172.21.15.107:0/44409257 wait complete. 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.503+0000 7f4e696e7740 1 -- start start 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 Processor -- start 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce095150 0x561cce09e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56964/0 (socket says 172.21.15.107:56964) 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e6588b640 1 -- 172.21.15.107:0/799963226 learned_addr learned my addr 172.21.15.107:0/799963226 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce09eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561ccde5d700 con 0x561cce094830 2024-08-20T06:38:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561ccdf817c0 con 0x561cce095150 2024-08-20T06:38:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561cce0a3de0 con 0x561cce0a10e0 2024-08-20T06:38:58.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.504+0000 7f4e64889640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce095150 0x561cce09e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 -- 172.21.15.107:0/799963226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 msgr2=0x561cce09eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce09eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e696e7740 10 client.0 _ll_register_callbacks cb 0x561ccdff46b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 -- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce095150 msgr2=0x561cce09e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce095150 0x561cce09e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cce0ad190 con 0x561cce094830 2024-08-20T06:38:58.510 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561ccdfaa100 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e6588b640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f80 tx=0x7f4e58004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60dfdf59900307fc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e48ff9640 10 client.15741 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e48ff9640 1 -- 172.21.15.107:0/799963226 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580103d0 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e48ff9640 1 -- 172.21.15.107:0/799963226 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4e58010a10 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e48ff9640 1 -- 172.21.15.107:0/799963226 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e58010c30 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e696e7740 10 client.15741 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.505+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561cce0af970 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e48ff9640 1 -- 172.21.15.107:0/799963226 <== mon.0 v2:172.21.15.107:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e58005040 con 0x561cce094830 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 10 client.15741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 20 client.15741 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0b1130 0x561cce0b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 4 client.15741 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cce00d830 con 0x561cce0b1130 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.506+0000 7f4e64889640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0b1130 0x561cce0b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.507+0000 7f4e64889640 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0b1130 0x561cce0b3590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ff71cec313dd17e server_cookie=530cfc6926ec0d23 in_seq=0 out_seq=0 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.507+0000 7f4e48ff9640 10 client.15741 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e48ff9640 1 -- 172.21.15.107:0/799963226 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cce00d830 con 0x561cce0b1130 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e48ff9640 10 client.15741 handle_command_reply: tid=0 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 client.15741 shutdown 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 msgr2=0x561cce09e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f80 tx=0x7f4e58004c70 comp rx=0 tx=0).stop 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 shutdown_connections 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0b1130 0x561cce0b3590 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce0a10e0 0x561cce09eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce095150 0x561cce09e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 --2- 172.21.15.107:0/799963226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce094830 0x561cce09e070 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 >> 172.21.15.107:0/799963226 conn(0x561cce092660 msgr2=0x561cce098200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e48ff9640 0 client.15741 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 shutdown_connections 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 1 -- 172.21.15.107:0/799963226 wait complete. 2024-08-20T06:38:58.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.508+0000 7f4e696e7740 20 client.15741 trim_cache size 0 max 16384 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 Processor -- start 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 -- start start 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce0989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce093dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0b8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601b0920 con 0x561cce0a10e0 2024-08-20T06:38:58.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ae1f0 con 0x561cce094830 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e696e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ac020 con 0x561cce095150 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0b8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0b8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:56970/0 (socket says 172.21.15.107:56970) 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e6588b640 1 -- 172.21.15.107:0/3371237869 learned_addr learned my addr 172.21.15.107:0/3371237869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e64889640 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce093dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.529+0000 7f4e6588b640 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce0989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 -- 172.21.15.107:0/3371237869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 msgr2=0x561cce0989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce0989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 -- 172.21.15.107:0/3371237869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 msgr2=0x561cce0b8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0b8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 -- 172.21.15.107:0/3371237869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cce0930e0 con 0x561cce095150 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e64889640 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce093dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009780 tx=0x7f4e5000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38599ad32421b8bd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e4affd640 1 -- 172.21.15.107:0/3371237869 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004c020 con 0x561cce095150 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e4affd640 1 -- 172.21.15.107:0/3371237869 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e500044d0 con 0x561cce095150 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.530+0000 7f4e4affd640 1 -- 172.21.15.107:0/3371237869 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50037040 con 0x561cce095150 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- 172.21.15.107:0/3371237869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 msgr2=0x561cce093dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce093dc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009780 tx=0x7f4e5000b040 comp rx=0 tx=0).stop 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- 172.21.15.107:0/3371237869 shutdown_connections 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0b8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce093dc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 --2- 172.21.15.107:0/3371237869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce0989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- 172.21.15.107:0/3371237869 >> 172.21.15.107:0/3371237869 conn(0x561cce092660 msgr2=0x561cce0936b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- 172.21.15.107:0/3371237869 shutdown_connections 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- 172.21.15.107:0/3371237869 wait complete. 2024-08-20T06:38:58.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 -- start start 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.531+0000 7f4e696e7740 1 Processor -- start 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce1151e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601ac6b0 con 0x561cce0a10e0 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601adb20 con 0x561cce094830 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601acd80 con 0x561cce095150 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e696e7740 10 client.0 _ll_register_callbacks cb 0x7f4e58015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e6588b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:40040/0 (socket says 172.21.15.107:40040) 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 -- 172.21.15.107:0/1863505548 learned_addr learned my addr 172.21.15.107:0/1863505548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:38:58.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e6588b640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce1151e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 -- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 msgr2=0x561cce1151e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce1151e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 -- 172.21.15.107:0/1863505548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 msgr2=0x561cce0a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0a5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 -- 172.21.15.107:0/1863505548 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cce0a5810 con 0x561cce095150 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e6588b640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.532+0000 7f4e64889640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a930 tx=0x7f4e50030780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2b896089e52a53 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e48ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5004c030 con 0x561cce095150 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e500044d0 con 0x561cce095150 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f4e50030e00 con 0x561cce095150 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e50037040 con 0x561cce095150 2024-08-20T06:38:58.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e696e7740 10 client.25177 fetch_fsmap learned FSMap version 25 2024-08-20T06:38:58.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561cce0a8c00 con 0x561cce095150 2024-08-20T06:38:58.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.533+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4e500044d0 con 0x561cce095150 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4e5005a030 con 0x561cce095150 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 10 client.25177 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 10 client.25177 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 20 client.25177 populate_metadata read hostname 'smithi107' 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0a9300 0x561cce0a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 4 client.25177 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:38:58.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561cce0092d0 con 0x561cce0a9300 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e6588b640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0a9300 0x561cce0a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e48ff9640 10 client.25177 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e6588b640 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0a9300 0x561cce0a9710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=853cf88d36d253f8 server_cookie=aed7c4b5740a7d80 in_seq=0 out_seq=0 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e48ff9640 1 -- 172.21.15.107:0/1863505548 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561cce0092d0 con 0x561cce0a9300 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.534+0000 7f4e48ff9640 10 client.25177 handle_command_reply: tid=0 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 client.25177 shutdown 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 msgr2=0x561cce115740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4e5002a930 tx=0x7f4e50030780 comp rx=0 tx=0).stop 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 shutdown_connections 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561cce0a10e0 0x561cce0a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x561cce0a9300 0x561cce0a9710 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561cce095150 0x561cce115740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e48ff9640 0 client.25177 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 --2- 172.21.15.107:0/1863505548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561cce094830 0x561cce1151e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 >> 172.21.15.107:0/1863505548 conn(0x561cce092660 msgr2=0x561cce09f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 shutdown_connections 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 1 -- 172.21.15.107:0/1863505548 wait complete. 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.535+0000 7f4e696e7740 20 client.25177 trim_cache size 0 max 16384 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:38:58.542 INFO:teuthology.orchestra.run.smithi107.stdout: "c1408381-e30c-42d7-83ed-686af9a0a0e2": { 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "c1408381-e30c-42d7-83ed-686af9a0a0e2", 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:38:58.543 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 msgr2=0x7f4e4c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 0x7f4e4c0b6090 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0a8140 tx=0x7f4e5c000bc0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 msgr2=0x7f4e60197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9c0 tx=0x7f4e5c00ee80 comp rx=0 tx=0).stop 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 shutdown_connections 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f4e60107d20 0x7f4e6019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f4e4c0b3bd0 0x7f4e4c0b6090 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e601073a0 0x7f4e601980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 --2- 172.21.15.107:0/2379288112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f4e60101990 0x7f4e60197b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:38:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 >> 172.21.15.107:0/2379288112 conn(0x7f4e600fd400 msgr2=0x7f4e600ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:38:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 shutdown_connections 2024-08-20T06:38:58.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:38:58.536+0000 7f4e672ee640 1 -- 172.21.15.107:0/2379288112 wait complete. 2024-08-20T06:38:58.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:38:58 smithi183 ceph-mon[22204]: pgmap v391: 97 pgs: 97 active+clean; 91 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-08-20T06:38:59.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:38:58 smithi193 ceph-mon[21944]: pgmap v391: 97 pgs: 97 active+clean; 91 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-08-20T06:38:59.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:38:58 smithi107 ceph-mon[18495]: pgmap v391: 97 pgs: 97 active+clean; 91 MiB data, 724 MiB used, 1.0 TiB / 1.0 TiB avail; 664 KiB/s rd, 1.9 MiB/s wr, 118 op/s 2024-08-20T06:38:59.258 INFO:tasks.cephfs.filesystem:scrub status for tag:c1408381-e30c-42d7-83ed-686af9a0a0e2 - {'path': '/', 'tag': 'c1408381-e30c-42d7-83ed-686af9a0a0e2', 'options': 'recursive,force'} 2024-08-20T06:39:00.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:00 smithi183 ceph-mon[22204]: pgmap v392: 97 pgs: 97 active+clean; 92 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-08-20T06:39:00.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:00 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:39:01.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:00 smithi193 ceph-mon[21944]: pgmap v392: 97 pgs: 97 active+clean; 92 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-08-20T06:39:01.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:00 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:39:01.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:00 smithi107 ceph-mon[18495]: pgmap v392: 97 pgs: 97 active+clean; 92 MiB data, 729 MiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.6 MiB/s wr, 102 op/s 2024-08-20T06:39:01.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:00 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:39:02.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:02 smithi183 ceph-mon[22204]: pgmap v393: 97 pgs: 97 active+clean; 93 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-08-20T06:39:03.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:02 smithi193 ceph-mon[21944]: pgmap v393: 97 pgs: 97 active+clean; 93 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-08-20T06:39:03.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:02 smithi107 ceph-mon[18495]: pgmap v393: 97 pgs: 97 active+clean; 93 MiB data, 737 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.5 MiB/s wr, 98 op/s 2024-08-20T06:39:04.985 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:04 smithi183 ceph-mon[22204]: pgmap v394: 97 pgs: 97 active+clean; 97 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-08-20T06:39:05.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:04 smithi193 ceph-mon[21944]: pgmap v394: 97 pgs: 97 active+clean; 97 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-08-20T06:39:05.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:04 smithi107 ceph-mon[18495]: pgmap v394: 97 pgs: 97 active+clean; 97 MiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-08-20T06:39:05.885 INFO:tasks.workunit.client.0.smithi107.stdout: 21 16248 177 10907 115 4214 20 2024-08-20T06:39:07.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:06 smithi193 ceph-mon[21944]: pgmap v395: 97 pgs: 97 active+clean; 98 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T06:39:07.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:06 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:07.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:06 smithi107 ceph-mon[18495]: pgmap v395: 97 pgs: 97 active+clean; 98 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T06:39:07.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:06 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:07.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:06 smithi183 ceph-mon[22204]: pgmap v395: 97 pgs: 97 active+clean; 98 MiB data, 750 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T06:39:07.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:06 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:09.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:08 smithi193 ceph-mon[21944]: pgmap v396: 97 pgs: 97 active+clean; 100 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 86 op/s 2024-08-20T06:39:09.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:08 smithi107 ceph-mon[18495]: pgmap v396: 97 pgs: 97 active+clean; 100 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 86 op/s 2024-08-20T06:39:09.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:08 smithi183 ceph-mon[22204]: pgmap v396: 97 pgs: 97 active+clean; 100 MiB data, 756 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 86 op/s 2024-08-20T06:39:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: pgmap v397: 97 pgs: 97 active+clean; 101 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1005 KiB/s wr, 70 op/s 2024-08-20T06:39:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:39:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.055 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:10 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: pgmap v397: 97 pgs: 97 active+clean; 101 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1005 KiB/s wr, 70 op/s 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:10 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: pgmap v397: 97 pgs: 97 active+clean; 101 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1005 KiB/s wr, 70 op/s 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:11.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:10 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:39:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:39:12.054 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:11 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:12.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:39:12.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:39:12.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:11 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:12.235 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:39:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:39:12.236 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:11 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:39:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:13 smithi107 ceph-mon[18495]: pgmap v398: 97 pgs: 97 active+clean; 102 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1006 KiB/s wr, 70 op/s 2024-08-20T06:39:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:13 smithi183 ceph-mon[22204]: pgmap v398: 97 pgs: 97 active+clean; 102 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1006 KiB/s wr, 70 op/s 2024-08-20T06:39:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:13 smithi193 ceph-mon[21944]: pgmap v398: 97 pgs: 97 active+clean; 102 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1006 KiB/s wr, 70 op/s 2024-08-20T06:39:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:15 smithi107 ceph-mon[18495]: pgmap v399: 97 pgs: 97 active+clean; 105 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-08-20T06:39:15.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:15 smithi183 ceph-mon[22204]: pgmap v399: 97 pgs: 97 active+clean; 105 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-08-20T06:39:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:15 smithi193 ceph-mon[21944]: pgmap v399: 97 pgs: 97 active+clean; 105 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.1 MiB/s wr, 89 op/s 2024-08-20T06:39:15.885 INFO:tasks.workunit.client.0.smithi107.stdout: 23 17068 95 10793 135 3989 28 2024-08-20T06:39:17.356 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:17 smithi193 ceph-mon[21944]: pgmap v400: 97 pgs: 97 active+clean; 106 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 880 KiB/s wr, 72 op/s 2024-08-20T06:39:17.356 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:17 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:17 smithi107 ceph-mon[18495]: pgmap v400: 97 pgs: 97 active+clean; 106 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 880 KiB/s wr, 72 op/s 2024-08-20T06:39:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:17 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:17 smithi183 ceph-mon[22204]: pgmap v400: 97 pgs: 97 active+clean; 106 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 880 KiB/s wr, 72 op/s 2024-08-20T06:39:17.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:17 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:39:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:19 smithi107 ceph-mon[18495]: pgmap v401: 97 pgs: 97 active+clean; 108 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 979 KiB/s wr, 86 op/s 2024-08-20T06:39:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:19 smithi183 ceph-mon[22204]: pgmap v401: 97 pgs: 97 active+clean; 108 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 979 KiB/s wr, 86 op/s 2024-08-20T06:39:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:19 smithi193 ceph-mon[21944]: pgmap v401: 97 pgs: 97 active+clean; 108 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 979 KiB/s wr, 86 op/s 2024-08-20T06:39:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:21 smithi107 ceph-mon[18495]: pgmap v402: 97 pgs: 97 active+clean; 109 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 824 KiB/s wr, 81 op/s 2024-08-20T06:39:21.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:21 smithi183 ceph-mon[22204]: pgmap v402: 97 pgs: 97 active+clean; 109 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 824 KiB/s wr, 81 op/s 2024-08-20T06:39:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:21 smithi193 ceph-mon[21944]: pgmap v402: 97 pgs: 97 active+clean; 109 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 824 KiB/s wr, 81 op/s 2024-08-20T06:39:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:23 smithi107 ceph-mon[18495]: pgmap v403: 97 pgs: 97 active+clean; 110 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 842 KiB/s wr, 86 op/s 2024-08-20T06:39:23.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:23 smithi183 ceph-mon[22204]: pgmap v403: 97 pgs: 97 active+clean; 110 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 842 KiB/s wr, 86 op/s 2024-08-20T06:39:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:23 smithi193 ceph-mon[21944]: pgmap v403: 97 pgs: 97 active+clean; 110 MiB data, 805 MiB used, 1.0 TiB / 1.0 TiB avail; 203 KiB/s rd, 842 KiB/s wr, 86 op/s 2024-08-20T06:39:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:25 smithi107 ceph-mon[18495]: pgmap v404: 97 pgs: 97 active+clean; 113 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-08-20T06:39:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:25 smithi183 ceph-mon[22204]: pgmap v404: 97 pgs: 97 active+clean; 113 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-08-20T06:39:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:25 smithi193 ceph-mon[21944]: pgmap v404: 97 pgs: 97 active+clean; 113 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.0 MiB/s wr, 107 op/s 2024-08-20T06:39:25.885 INFO:tasks.workunit.client.0.smithi107.stdout: 27 17384 175 10402 90 2688 37 2024-08-20T06:39:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:27 smithi107 ceph-mon[18495]: pgmap v405: 97 pgs: 97 active+clean; 114 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 839 KiB/s wr, 84 op/s 2024-08-20T06:39:27.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:27 smithi183 ceph-mon[22204]: pgmap v405: 97 pgs: 97 active+clean; 114 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 839 KiB/s wr, 84 op/s 2024-08-20T06:39:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:27 smithi193 ceph-mon[21944]: pgmap v405: 97 pgs: 97 active+clean; 114 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 839 KiB/s wr, 84 op/s 2024-08-20T06:39:29.259 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:39:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:29 smithi107 ceph-mon[18495]: pgmap v406: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-08-20T06:39:29.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:29 smithi183 ceph-mon[22204]: pgmap v406: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-08-20T06:39:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:29 smithi193 ceph-mon[21944]: pgmap v406: 97 pgs: 97 active+clean; 117 MiB data, 795 MiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.0 MiB/s wr, 101 op/s 2024-08-20T06:39:29.641 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:31 smithi107 ceph-mon[18495]: pgmap v407: 97 pgs: 97 active+clean; 119 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-08-20T06:39:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:31 smithi183 ceph-mon[22204]: pgmap v407: 97 pgs: 97 active+clean; 119 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-08-20T06:39:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:31 smithi193 ceph-mon[21944]: pgmap v407: 97 pgs: 97 active+clean; 119 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.0 MiB/s wr, 98 op/s 2024-08-20T06:39:32.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 -- 172.21.15.107:0/4145766634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 msgr2=0x7f30a4101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 --2- 172.21.15.107:0/4145766634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a4101590 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3094009900 tx=0x7f309402a410 comp rx=0 tx=0).stop 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 -- 172.21.15.107:0/4145766634 shutdown_connections 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 --2- 172.21.15.107:0/4145766634 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a4103e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 --2- 172.21.15.107:0/4145766634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 --2- 172.21.15.107:0/4145766634 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a4101590 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 -- 172.21.15.107:0/4145766634 >> 172.21.15.107:0/4145766634 conn(0x7f30a4076bd0 msgr2=0x7f30a4076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 -- 172.21.15.107:0/4145766634 shutdown_connections 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.291+0000 7f30aa81a640 1 -- 172.21.15.107:0/4145766634 wait complete. 2024-08-20T06:39:32.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 Processor -- start 2024-08-20T06:39:32.294 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 -- start start 2024-08-20T06:39:32.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.295 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4112960 con 0x7f30a4069f20 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41127e0 con 0x7f30a4101f30 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30aa81a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30a4112ae0 con 0x7f30a4101ad0 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44652/0 (socket says 172.21.15.107:44652) 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.292+0000 7f30a8db7640 1 -- 172.21.15.107:0/2542933791 learned_addr learned my addr 172.21.15.107:0/2542933791 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a37fe640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 -- 172.21.15.107:0/2542933791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 msgr2=0x7f30a419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 -- 172.21.15.107:0/2542933791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 msgr2=0x7f30a41a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 -- 172.21.15.107:0/2542933791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a41a1200 con 0x7f30a4069f20 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a8db7640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a37fe640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.293+0000 7f30a3fff640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a419b150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3094009850 tx=0x7f3094003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.294+0000 7f30a17fa640 1 -- 172.21.15.107:0/2542933791 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309403f070 con 0x7f30a4069f20 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.294+0000 7f30a17fa640 1 -- 172.21.15.107:0/2542933791 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f309403baf0 con 0x7f30a4069f20 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.294+0000 7f30a17fa640 1 -- 172.21.15.107:0/2542933791 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094035030 con 0x7f30a4069f20 2024-08-20T06:39:32.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.294+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a41a1490 con 0x7f30a4069f20 2024-08-20T06:39:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.294+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a41a7bb0 con 0x7f30a4069f20 2024-08-20T06:39:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.297+0000 7f30acc13740 1 Processor -- start 2024-08-20T06:39:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.297+0000 7f30a17fa640 1 -- 172.21.15.107:0/2542933791 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f3094042070 con 0x7f30a4069f20 2024-08-20T06:39:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 -- start start 2024-08-20T06:39:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738282ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 0x55b7382854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b7381ed8e0 con 0x55b738277150 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7381dcde0 con 0x55b738276830 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30acc13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b7381e0e20 con 0x55b7382830e0 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30a17fa640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 0x7f308c0b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44662/0 (socket says 172.21.15.107:44662) 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.298+0000 7f30a8db7640 1 -- 172.21.15.107:0/1579725198 learned_addr learned my addr 172.21.15.107:0/1579725198 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a17fa640 1 -- 172.21.15.107:0/2542933791 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3094104260 con 0x7f30a4069f20 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a37fe640 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 0x55b7382854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 -- 172.21.15.107:0/1579725198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 msgr2=0x55b7382854d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 0x55b7382854d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 -- 172.21.15.107:0/1579725198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 msgr2=0x55b738282ba0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738282ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 -- 172.21.15.107:0/1579725198 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b737f3c650 con 0x55b738276830 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a37fe640 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 0x55b7382854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:39:32.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 0x7f308c0b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f30a8db7640 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f309c009a40 tx=0x7f309c00fe10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516e058956a4505f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.299+0000 7f308a7fc640 1 -- 172.21.15.107:0/1579725198 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c01c000 con 0x55b738276830 2024-08-20T06:39:32.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f308a7fc640 1 -- 172.21.15.107:0/1579725198 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f309c0108c0 con 0x55b738276830 2024-08-20T06:39:32.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f308a7fc640 1 -- 172.21.15.107:0/1579725198 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c00e710 con 0x55b738276830 2024-08-20T06:39:32.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 -- 172.21.15.107:0/1579725198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 msgr2=0x55b738276c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30a8db7640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 0x7f308c0b6150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f309c022000 tx=0x7f309c01f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f309c009a40 tx=0x7f309c00fe10 comp rx=0 tx=0).stop 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 -- 172.21.15.107:0/1579725198 shutdown_connections 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b7382830e0 0x55b7382854d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738282ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 --2- 172.21.15.107:0/1579725198 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b738276c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.300+0000 7f30acc13740 1 -- 172.21.15.107:0/1579725198 >> 172.21.15.107:0/1579725198 conn(0x55b738274660 msgr2=0x55b738275800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.301+0000 7f30acc13740 1 -- 172.21.15.107:0/1579725198 shutdown_connections 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.301+0000 7f30acc13740 1 -- 172.21.15.107:0/1579725198 wait complete. 2024-08-20T06:39:32.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.301+0000 7f30acc13740 1 -- start start 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.301+0000 7f30acc13740 1 Processor -- start 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738276830 0x55b7382800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b7382805e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b7382805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44666/0 (socket says 172.21.15.107:44666) 2024-08-20T06:39:32.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 -- 172.21.15.107:0/4086111887 learned_addr learned my addr 172.21.15.107:0/4086111887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b73803f700 con 0x55b738276830 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b7381637c0 con 0x55b7382830e0 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a3fff640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738276830 0x55b7382800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b738285d90 con 0x55b738277150 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30acc13740 10 client.0 _ll_register_callbacks cb 0x55b7381d66b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 -- 172.21.15.107:0/4086111887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 msgr2=0x55b7382805e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b7382805e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 -- 172.21.15.107:0/4086111887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738276830 msgr2=0x55b7382800a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738276830 0x55b7382800a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.302+0000 7f30a8db7640 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73828f0f0 con 0x55b7382830e0 2024-08-20T06:39:32.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30a8db7640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f309c00f520 tx=0x7f309c00ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36c88cee7643e12 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30837fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c004980 con 0x55b7382830e0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b73818c100 con 0x55b7382830e0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f309c00ed60 con 0x55b7382830e0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f309c010430 con 0x55b7382830e0 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30acc13740 10 client.25274 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:32.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.303+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b738291a80 con 0x55b7382830e0 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b73818c100 con 0x55b7382830e0 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f309c036450 con 0x55b7382830e0 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 10 client.25274 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 10 client.25274 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 20 client.25274 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:32.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 4 client.25274 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7381ef830 con 0x55b738292dd0 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30a3fff640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30837fe640 10 client.25274 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.304+0000 7f30a3fff640 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382951c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbc46368e5a6d8db server_cookie=a1c99b5a6ec8a928 in_seq=0 out_seq=0 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30837fe640 1 -- 172.21.15.107:0/4086111887 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b7381ef830 con 0x55b738292dd0 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30837fe640 10 client.25274 handle_command_reply: tid=0 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 client.25274 shutdown 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 msgr2=0x55b738280b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f309c00f520 tx=0x7f309c00ec70 comp rx=0 tx=0).stop 2024-08-20T06:39:32.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 shutdown_connections 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b738280b20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382951c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b7382805e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30837fe640 0 client.25274 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.305+0000 7f30acc13740 1 --2- 172.21.15.107:0/4086111887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738276830 0x55b7382800a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.306+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 >> 172.21.15.107:0/4086111887 conn(0x55b738274660 msgr2=0x55b73827a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.306+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 shutdown_connections 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.306+0000 7f30acc13740 1 -- 172.21.15.107:0/4086111887 wait complete. 2024-08-20T06:39:32.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.306+0000 7f30acc13740 20 client.25274 trim_cache size 0 max 16384 2024-08-20T06:39:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.329+0000 7f30acc13740 1 Processor -- start 2024-08-20T06:39:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.329+0000 7f30acc13740 1 -- start start 2024-08-20T06:39:32.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.329+0000 7f30acc13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b73827a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.329+0000 7f30acc13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b73827f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.329+0000 7f30acc13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 0x55b73829aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30acc13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b2c00 con 0x55b7382830e0 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30acc13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41b04d0 con 0x55b738276830 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30acc13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ae300 con 0x55b738277150 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 0x55b73829aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b73827a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a8db7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 0x55b73829aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55772/0 (socket says 172.21.15.107:55772) 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a8db7640 1 -- 172.21.15.107:0/1218719247 learned_addr learned my addr 172.21.15.107:0/1218719247 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a37fe640 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b73827f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 -- 172.21.15.107:0/1218719247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 msgr2=0x55b73827f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b73827f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 -- 172.21.15.107:0/1218719247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 msgr2=0x55b73829aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 0x55b73829aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 -- 172.21.15.107:0/1218719247 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b738276f80 con 0x55b738276830 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f30a3fff640 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b73827a9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b7382799b0 tx=0x7f30940057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374b392c7a010861 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f3089ffb640 1 -- 172.21.15.107:0/1218719247 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094049040 con 0x55b738276830 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f3089ffb640 1 -- 172.21.15.107:0/1218719247 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f30940d4260 con 0x55b738276830 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.330+0000 7f3089ffb640 1 -- 172.21.15.107:0/1218719247 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30940c7440 con 0x55b738276830 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 -- 172.21.15.107:0/1218719247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 msgr2=0x55b73827a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b73827a9f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55b7382799b0 tx=0x7f30940057c0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 -- 172.21.15.107:0/1218719247 shutdown_connections 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b7382830e0 0x55b73829aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738277150 0x55b73827f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 --2- 172.21.15.107:0/1218719247 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b738276830 0x55b73827a9f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 -- 172.21.15.107:0/1218719247 >> 172.21.15.107:0/1218719247 conn(0x55b738274660 msgr2=0x55b738281de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 -- 172.21.15.107:0/1218719247 shutdown_connections 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.331+0000 7f30acc13740 1 -- 172.21.15.107:0/1218719247 wait complete. 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.332+0000 7f30acc13740 1 -- start start 2024-08-20T06:39:32.335 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.332+0000 7f30acc13740 1 Processor -- start 2024-08-20T06:39:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738289290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b7382898a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41ae990 con 0x55b738277150 2024-08-20T06:39:32.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41afe00 con 0x55b7382830e0 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a37fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738289290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f30a41af060 con 0x55b738276830 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30acc13740 10 client.0 _ll_register_callbacks cb 0x7f309c00f860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:58770/0 (socket says 172.21.15.107:58770) 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a3fff640 1 -- 172.21.15.107:0/3733765536 learned_addr learned my addr 172.21.15.107:0/3733765536 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b7382898a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 -- 172.21.15.107:0/3733765536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 msgr2=0x55b73828b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 -- 172.21.15.107:0/3733765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 msgr2=0x55b738289290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738289290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a8db7640 1 -- 172.21.15.107:0/3733765536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b73828a0d0 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a37fe640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738289290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.333+0000 7f30a3fff640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30a8db7640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b7382898a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f309c002230 tx=0x7f309c00f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4989ec87d5e907b2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30837fe640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309c00ea00 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f309c01be00 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30940d4260 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f309c036450 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30acc13740 10 client.25280 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b7382fcc00 con 0x55b7382830e0 2024-08-20T06:39:32.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.334+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f309c031070 con 0x55b7382830e0 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f309c010840 con 0x55b7382830e0 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 10 client.25280 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 20 client.25280 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 4 client.25280 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f30940d4260 con 0x55b738292dd0 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30a3fff640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30837fe640 10 client.25280 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.335+0000 7f30a3fff640 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382ff420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e24ff2a255924a server_cookie=62b937e7d1274ced in_seq=0 out_seq=0 2024-08-20T06:39:32.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30837fe640 1 -- 172.21.15.107:0/3733765536 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f30940d4260 con 0x55b738292dd0 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30837fe640 10 client.25280 handle_command_reply: tid=0 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 client.25280 shutdown 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 msgr2=0x55b7382898a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b7382898a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f309c002230 tx=0x7f309c00f640 comp rx=0 tx=0).stop 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 shutdown_connections 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b7382830e0 0x55b7382898a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b738277150 0x55b738289290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55b738292dd0 0x55b7382ff420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 --2- 172.21.15.107:0/3733765536 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b738276830 0x55b73828b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 >> 172.21.15.107:0/3733765536 conn(0x55b738274660 msgr2=0x55b738281500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30837fe640 0 client.25280 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 shutdown_connections 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 1 -- 172.21.15.107:0/3733765536 wait complete. 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.336+0000 7f30acc13740 20 client.25280 trim_cache size 0 max 16384 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:39:32.343 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 msgr2=0x7f308c0b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 0x7f308c0b6150 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f309c022000 tx=0x7f309c01f000 comp rx=0 tx=0).stop 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 msgr2=0x7f30a419b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a419b150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3094009850 tx=0x7f3094003c00 comp rx=0 tx=0).stop 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 shutdown_connections 2024-08-20T06:39:32.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f30a4101f30 0x7f30a41a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f308c0b3c90 0x7f308c0b6150 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30a4101ad0 0x7f30a419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 --2- 172.21.15.107:0/2542933791 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f30a4069f20 0x7f30a419b150 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 >> 172.21.15.107:0/2542933791 conn(0x7f30a4076bd0 msgr2=0x7f30a41028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 shutdown_connections 2024-08-20T06:39:32.345 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:32.337+0000 7f30aa81a640 1 -- 172.21.15.107:0/2542933791 wait complete. 2024-08-20T06:39:33.230 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:33 smithi107 ceph-mon[18495]: pgmap v408: 97 pgs: 97 active+clean; 120 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-08-20T06:39:33.231 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:39:33.231 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:39:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:33 smithi183 ceph-mon[22204]: pgmap v408: 97 pgs: 97 active+clean; 120 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-08-20T06:39:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:33 smithi193 ceph-mon[21944]: pgmap v408: 97 pgs: 97 active+clean; 120 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.1 MiB/s wr, 97 op/s 2024-08-20T06:39:33.577 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 -- 172.21.15.107:0/951926183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0081026a0 msgr2=0x7fe008102a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 --2- 172.21.15.107:0/951926183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0081026a0 0x7fe008102a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdffc009930 tx=0x7fdffc02a420 comp rx=0 tx=0).stop 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 -- 172.21.15.107:0/951926183 shutdown_connections 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 --2- 172.21.15.107:0/951926183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe008107e10 0x7fe00810e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 --2- 172.21.15.107:0/951926183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107490 0x7fe0081078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 --2- 172.21.15.107:0/951926183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe0081026a0 0x7fe008102a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.107+0000 7fe00f08d640 1 -- 172.21.15.107:0/951926183 >> 172.21.15.107:0/951926183 conn(0x7fe008076bd0 msgr2=0x7fe008076fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.108+0000 7fe00f08d640 1 -- 172.21.15.107:0/951926183 shutdown_connections 2024-08-20T06:39:34.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.108+0000 7fe00f08d640 1 -- 172.21.15.107:0/951926183 wait complete. 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 Processor -- start 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 -- start start 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 0x7fe008108630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 0x7fe008108b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe008111880 con 0x7fe008107e10 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe008111700 con 0x7fe008107490 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00f08d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe008111a00 con 0x7fe0081026a0 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe007fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 0x7fe008108b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:34.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00d62a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00d62a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55786/0 (socket says 172.21.15.107:55786) 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00d62a640 1 -- 172.21.15.107:0/1359633430 learned_addr learned my addr 172.21.15.107:0/1359633430 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.109+0000 7fe00ce29640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 0x7fe008108630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 -- 172.21.15.107:0/1359633430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 msgr2=0x7fe008108630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 0x7fe008108630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 -- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 msgr2=0x7fe0081090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 -- 172.21.15.107:0/1359633430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0081a7340 con 0x7fe008107490 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe00d62a640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe00ce29640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 0x7fe008108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:34.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe007fff640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 0x7fe008108b70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff400ccc0 con 0x7fe008107490 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdff40108c0 con 0x7fe008107490 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff40187e0 con 0x7fe008107490 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.110+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0081a7630 con 0x7fe008107490 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.111+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0081a7bc0 con 0x7fe008107490 2024-08-20T06:39:34.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.112+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe008102a80 con 0x7fe008107490 2024-08-20T06:39:34.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.113+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fdff4010430 con 0x7fe008107490 2024-08-20T06:39:34.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.114+0000 7fe005ffb640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 0x7fdff80b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:34.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.114+0000 7fe00ce29640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 0x7fdff80b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:34.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.115+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdff40e3e90 con 0x7fe008107490 2024-08-20T06:39:34.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.115+0000 7fe00ce29640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 0x7fdff80b6150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0057c0 tx=0x7fdffc02a920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:34.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.119+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdff40aae50 con 0x7fe008107490 2024-08-20T06:39:34.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.485+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe0081a7f80 con 0x7fe008107490 2024-08-20T06:39:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.486+0000 7fe005ffb640 1 -- 172.21.15.107:0/1359633430 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7fdff40af3a0 con 0x7fe008107490 2024-08-20T06:39:34.487 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:39:34.487 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-08-20T06:39:16:059531+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:39:15.312343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:39:34.487 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-08-20T06:39:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 msgr2=0x7fdff80b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 0x7fdff80b6150 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdffc0057c0 tx=0x7fdffc02a920 comp rx=0 tx=0).stop 2024-08-20T06:39:34.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 msgr2=0x7fe008108b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 0x7fe008108b70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdff400e9f0 tx=0x7fdff400eeb0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 shutdown_connections 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe008107e10 0x7fe0081090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe008107490 0x7fe008108b70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdff80b3c90 0x7fdff80b6150 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 --2- 172.21.15.107:0/1359633430 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe0081026a0 0x7fe008108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 >> 172.21.15.107:0/1359633430 conn(0x7fe008076bd0 msgr2=0x7fe008105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 shutdown_connections 2024-08-20T06:39:34.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:34.488+0000 7fe00f08d640 1 -- 172.21.15.107:0/1359633430 wait complete. 2024-08-20T06:39:35.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:39:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:35 smithi107 ceph-mon[18495]: pgmap v409: 97 pgs: 97 active+clean; 126 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-08-20T06:39:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:35 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1359633430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:39:35.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:35 smithi183 ceph-mon[22204]: pgmap v409: 97 pgs: 97 active+clean; 126 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-08-20T06:39:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:35 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1359633430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:39:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:35 smithi193 ceph-mon[21944]: pgmap v409: 97 pgs: 97 active+clean; 126 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 118 op/s 2024-08-20T06:39:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1359633430' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:39:35.610 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:35.885 INFO:tasks.workunit.client.0.smithi107.stdout: 31 15879 253 9517 187 2374 51 2024-08-20T06:39:36.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1954362173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 msgr2=0x7f3e4410f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1954362173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e4410f610 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009900 tx=0x7f3e3802a410 comp rx=0 tx=0).stop 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1954362173 shutdown_connections 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1954362173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e4410f610 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1954362173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1954362173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e44100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.144+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1954362173 >> 172.21.15.107:0/1954362173 conn(0x7f3e440fc730 msgr2=0x7f3e440feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.145+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1954362173 shutdown_connections 2024-08-20T06:39:36.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.145+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1954362173 wait complete. 2024-08-20T06:39:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.145+0000 7f3e4ba33640 1 Processor -- start 2024-08-20T06:39:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.145+0000 7f3e4ba33640 1 -- start start 2024-08-20T06:39:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e4419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e441a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112980 con 0x7f3e44107d20 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112800 con 0x7f3e44100a20 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e497cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e4ba33640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112b00 con 0x7f3e441073a0 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e497cf640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43726/0 (socket says 172.21.15.107:43726) 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e497cf640 1 -- 172.21.15.107:0/1893690466 learned_addr learned my addr 172.21.15.107:0/1893690466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e441a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.146+0000 7f3e48fce640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e4419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 -- 172.21.15.107:0/1893690466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 msgr2=0x7f3e4419b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e4419b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 -- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 msgr2=0x7f3e441a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e441a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 -- 172.21.15.107:0/1893690466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a1180 con 0x7f3e44100a20 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e441a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e48fce640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e4419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e497cf640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e9f0 tx=0x7f3e3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e427fc640 1 -- 172.21.15.107:0/1893690466 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c00cd80 con 0x7f3e44100a20 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a1470 con 0x7f3e44100a20 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a7d00 con 0x7f3e44100a20 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e427fc640 1 -- 172.21.15.107:0/1893690466 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e3c0108c0 con 0x7f3e44100a20 2024-08-20T06:39:36.150 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.147+0000 7f3e427fc640 1 -- 172.21.15.107:0/1893690466 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0187c0 con 0x7f3e44100a20 2024-08-20T06:39:36.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.150+0000 7f3e427fc640 1 -- 172.21.15.107:0/1893690466 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f3e3c010430 con 0x7f3e44100a20 2024-08-20T06:39:36.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.151+0000 7f3e4de2c740 1 Processor -- start 2024-08-20T06:39:36.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.151+0000 7f3e4de2c740 1 -- start start 2024-08-20T06:39:36.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.151+0000 7f3e427fc640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 0x7f3e2c0be920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.151+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 0x7f3e2c0be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.151+0000 7f3e427fc640 1 -- 172.21.15.107:0/1893690466 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3e3c0e3d80 con 0x7f3e44100a20 2024-08-20T06:39:36.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca9294bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e497cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e497cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41408/0 (socket says 172.21.15.107:41408) 2024-08-20T06:39:36.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e497cf640 1 -- 172.21.15.107:0/998833560 learned_addr learned my addr 172.21.15.107:0/998833560 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:36.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9297520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e48fce640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca9294bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 0x7f3e2c0be920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006010 tx=0x7f3e38005bd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ca91ff930 con 0x563ca9288880 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563ca91eee30 con 0x563ca92891a0 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ca91f2e70 con 0x563ca9295130 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.152+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9297520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 -- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 msgr2=0x563ca9297520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9297520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 -- 172.21.15.107:0/998833560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 msgr2=0x563ca9294bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca9294bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 -- 172.21.15.107:0/998833560 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca8f4e650 con 0x563ca9288880 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9297520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:39:36.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e497cf640 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0a6e00 tx=0x7f3e3c010820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897eefce336a1420 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e2b7fe640 1 -- 172.21.15.107:0/998833560 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0b3d80 con 0x563ca9288880 2024-08-20T06:39:36.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e2b7fe640 1 -- 172.21.15.107:0/998833560 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e3c0a7370 con 0x563ca9288880 2024-08-20T06:39:36.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.153+0000 7f3e2b7fe640 1 -- 172.21.15.107:0/998833560 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0a6950 con 0x563ca9288880 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 msgr2=0x563ca9288c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0a6e00 tx=0x7f3e3c010820 comp rx=0 tx=0).stop 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 shutdown_connections 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9297520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca9294bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/998833560 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9288880 0x563ca9288c60 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 >> 172.21.15.107:0/998833560 conn(0x563ca92866b0 msgr2=0x563ca9287850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.154+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 shutdown_connections 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.155+0000 7f3e4de2c740 1 -- 172.21.15.107:0/998833560 wait complete. 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.155+0000 7f3e4de2c740 1 -- start start 2024-08-20T06:39:36.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.155+0000 7f3e4de2c740 1 Processor -- start 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca92920b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 0x563ca92925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9292b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x563ca9051700 con 0x563ca92891a0 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x563ca9175810 con 0x563ca9288880 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x563ca9297d20 con 0x563ca9295130 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e4de2c740 10 client.0 _ll_register_callbacks cb 0x563ca91e8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 0x563ca92925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9292b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 0x563ca92925f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41412/0 (socket says 172.21.15.107:41412) 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e48fce640 1 -- 172.21.15.107:0/2983671097 learned_addr learned my addr 172.21.15.107:0/2983671097 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:36.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca92920b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 -- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 msgr2=0x563ca92920b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca92920b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 -- 172.21.15.107:0/2983671097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 msgr2=0x563ca92925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 0x563ca92925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.156+0000 7f3e49fd0640 1 -- 172.21.15.107:0/2983671097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca92a1130 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9292b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e380057c0 tx=0x7f3e3802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddea9190ae4ebb09 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38035050 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3e380037f0 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ca919e150 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e380039f0 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e4de2c740 10 client.25207 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ca92a3ac0 con 0x563ca9295130 2024-08-20T06:39:36.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.157+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x563ca919e150 con 0x563ca9295130 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e38052b30 con 0x563ca9295130 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 10 client.25207 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 10 client.25207 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 20 client.25207 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca92a5020 0x563ca92a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 4 client.25207 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ca9201880 con 0x563ca92a5020 2024-08-20T06:39:36.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca92a5020 0x563ca92a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.158+0000 7f3e28ff9640 10 client.25207 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:39:36.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca92a5020 0x563ca92a7440 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aa67e412ebc0627a server_cookie=5dbab1e6a1020884 in_seq=0 out_seq=0 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e28ff9640 1 -- 172.21.15.107:0/2983671097 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ca9201880 con 0x563ca92a5020 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e28ff9640 10 client.25207 handle_command_reply: tid=0 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 client.25207 shutdown 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 msgr2=0x563ca9292b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9292b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3e380057c0 tx=0x7f3e3802a410 comp rx=0 tx=0).stop 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 shutdown_connections 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9295130 0x563ca9292b30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca92891a0 0x563ca92925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.159+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca92a5020 0x563ca92a7440 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2983671097 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca92920b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e28ff9640 0 client.25207 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 >> 172.21.15.107:0/2983671097 conn(0x563ca92866b0 msgr2=0x563ca928c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 shutdown_connections 2024-08-20T06:39:36.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2983671097 wait complete. 2024-08-20T06:39:36.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.160+0000 7f3e4de2c740 20 client.25207 trim_cache size 0 max 16384 2024-08-20T06:39:36.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 Processor -- start 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 -- start start 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 0x563ca9291940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca92ad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e30010d90 con 0x563ca9295130 2024-08-20T06:39:36.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000e660 con 0x563ca92891a0 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000c490 con 0x563ca9288880 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43742/0 (socket says 172.21.15.107:43742) 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e48fce640 1 -- 172.21.15.107:0/2708051388 learned_addr learned my addr 172.21.15.107:0/2708051388 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e49fd0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca92ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.179+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 0x563ca9291940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 -- 172.21.15.107:0/2708051388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 msgr2=0x563ca92acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 -- 172.21.15.107:0/2708051388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 msgr2=0x563ca92ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca92ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 -- 172.21.15.107:0/2708051388 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ca928dc40 con 0x563ca9288880 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e48fce640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca92ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e497cf640 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 0x563ca9291940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0a7ff0 tx=0x7f3e3c0af450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db3fa1f8704a6cb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e2affd640 1 -- 172.21.15.107:0/2708051388 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0af040 con 0x563ca9288880 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e2affd640 1 -- 172.21.15.107:0/2708051388 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e3c010d50 con 0x563ca9288880 2024-08-20T06:39:36.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.180+0000 7f3e2affd640 1 -- 172.21.15.107:0/2708051388 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c0a66f0 con 0x563ca9288880 2024-08-20T06:39:36.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2708051388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 msgr2=0x563ca9291940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 0x563ca9291940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c0a7ff0 tx=0x7f3e3c0af450 comp rx=0 tx=0).stop 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2708051388 shutdown_connections 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca92ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca92891a0 0x563ca92acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/2708051388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca9288880 0x563ca9291940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2708051388 >> 172.21.15.107:0/2708051388 conn(0x563ca92866b0 msgr2=0x563ca9293f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.181+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2708051388 shutdown_connections 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.182+0000 7f3e4de2c740 1 -- 172.21.15.107:0/2708051388 wait complete. 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.182+0000 7f3e4de2c740 1 -- start start 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.182+0000 7f3e4de2c740 1 Processor -- start 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca9298400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca930dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000cb20 con 0x563ca9295130 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000df90 con 0x563ca9288880 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e48fce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:44480/0 (socket says 172.21.15.107:44480) 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e48fce640 1 -- 172.21.15.107:0/560943531 learned_addr learned my addr 172.21.15.107:0/560943531 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e497cf640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca9298400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca930dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3000d1f0 con 0x563ca92891a0 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.183+0000 7f3e4de2c740 10 client.0 _ll_register_callbacks cb 0x7f3e38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 -- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 msgr2=0x563ca9298400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca9298400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 -- 172.21.15.107:0/560943531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 msgr2=0x563ca930dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca930dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 -- 172.21.15.107:0/560943531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ca930e210 con 0x563ca92891a0 2024-08-20T06:39:36.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e49fd0640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca930dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e48fce640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e34011d10 tx=0x7f3e34012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd16deb67c97271d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34022060 con 0x563ca92891a0 2024-08-20T06:39:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3e340107e0 con 0x563ca92891a0 2024-08-20T06:39:36.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.184+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3c010d50 con 0x563ca92891a0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3e340109e0 con 0x563ca92891a0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 10 client.25213 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x563ca930f470 con 0x563ca92891a0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3e3c010d50 con 0x563ca92891a0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3e34027070 con 0x563ca92891a0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 10 client.25213 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 10 client.25213 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 20 client.25213 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca930fc50 0x563ca9312070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 4 client.25213 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.185+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563ca91fd320 con 0x563ca930fc50 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e497cf640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca930fc50 0x563ca9312070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:36.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e497cf640 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca930fc50 0x563ca9312070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af9b16311f323e7b server_cookie=b72adf66e2b911d9 in_seq=0 out_seq=0 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e28ff9640 10 client.25213 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e28ff9640 1 -- 172.21.15.107:0/560943531 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563ca91fd320 con 0x563ca930fc50 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e28ff9640 10 client.25213 handle_command_reply: tid=0 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 client.25213 shutdown 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 msgr2=0x563ca9298940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e34011d10 tx=0x7f3e34012820 comp rx=0 tx=0).stop 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 shutdown_connections 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x563ca9295130 0x563ca930dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563ca92891a0 0x563ca9298940 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3e34011d10 tx=0x7f3e34012820 comp rx=0 tx=0).stop 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x563ca930fc50 0x563ca9312070 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 --2- 172.21.15.107:0/560943531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x563ca9288880 0x563ca9298400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 >> 172.21.15.107:0/560943531 conn(0x563ca92866b0 msgr2=0x563ca9292020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.186+0000 7f3e28ff9640 0 client.25213 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:39:36.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.187+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 shutdown_connections 2024-08-20T06:39:36.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.187+0000 7f3e4de2c740 1 -- 172.21.15.107:0/560943531 wait complete. 2024-08-20T06:39:36.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.187+0000 7f3e4de2c740 20 client.25213 trim_cache size 0 max 16384 2024-08-20T06:39:36.192 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 msgr2=0x7f3e2c0be920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 0x7f3e2c0be920 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3e38006010 tx=0x7f3e38005bd0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 msgr2=0x7f3e4419b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00e9f0 tx=0x7f3e3c00eeb0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 shutdown_connections 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3e44107d20 0x7f3e441a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3e2c0bc460 0x7f3e2c0be920 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e441073a0 0x7f3e4419b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 --2- 172.21.15.107:0/1893690466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3e44100a20 0x7f3e4419b0d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 >> 172.21.15.107:0/1893690466 conn(0x7f3e440fc730 msgr2=0x7f3e4410d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 shutdown_connections 2024-08-20T06:39:36.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:36.188+0000 7f3e4ba33640 1 -- 172.21.15.107:0/1893690466 wait complete. 2024-08-20T06:39:36.948 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:39:37.282 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:37 smithi107 ceph-mon[18495]: pgmap v410: 97 pgs: 97 active+clean; 128 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T06:39:37.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:37 smithi183 ceph-mon[22204]: pgmap v410: 97 pgs: 97 active+clean; 128 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T06:39:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:37 smithi193 ceph-mon[21944]: pgmap v410: 97 pgs: 97 active+clean; 128 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T06:39:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:39 smithi107 ceph-mon[18495]: pgmap v411: 97 pgs: 97 active+clean; 133 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-08-20T06:39:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:39 smithi183 ceph-mon[22204]: pgmap v411: 97 pgs: 97 active+clean; 133 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-08-20T06:39:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:39 smithi193 ceph-mon[21944]: pgmap v411: 97 pgs: 97 active+clean; 133 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-08-20T06:39:40.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 -- 172.21.15.107:0/4266311351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c075210 msgr2=0x7ff52c0755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 --2- 172.21.15.107:0/4266311351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c075210 0x7ff52c0755f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff520009900 tx=0x7ff52002a410 comp rx=0 tx=0).stop 2024-08-20T06:39:40.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 -- 172.21.15.107:0/4266311351 shutdown_connections 2024-08-20T06:39:40.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 --2- 172.21.15.107:0/4266311351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c10f090 0x7ff52c111480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 --2- 172.21.15.107:0/4266311351 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 --2- 172.21.15.107:0/4266311351 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c075210 0x7ff52c0755f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 -- 172.21.15.107:0/4266311351 >> 172.21.15.107:0/4266311351 conn(0x7ff52c0fe550 msgr2=0x7ff52c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 -- 172.21.15.107:0/4266311351 shutdown_connections 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.134+0000 7ff533251640 1 -- 172.21.15.107:0/4266311351 wait complete. 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 Processor -- start 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 -- start start 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 0x7ff52c10e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 0x7ff52c10c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c114b20 con 0x7ff52c10f090 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1149a0 con 0x7ff52c075b30 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff533251640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c114ca0 con 0x7ff52c075210 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff530fed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 0x7ff52c10e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43780/0 (socket says 172.21.15.107:43780) 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff52bfff640 1 -- 172.21.15.107:0/4057495611 learned_addr learned my addr 172.21.15.107:0/4057495611 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.135+0000 7ff5317ee640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 0x7ff52c10c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 -- 172.21.15.107:0/4057495611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 msgr2=0x7ff52c10e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 0x7ff52c10e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 -- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 msgr2=0x7ff52c10c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 0x7ff52c10c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 -- 172.21.15.107:0/4057495611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1ab840 con 0x7ff52c075b30 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff5317ee640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 0x7ff52c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff530fed640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 0x7ff52c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff52bfff640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9f0 tx=0x7ff51800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff529ffb640 1 -- 172.21.15.107:0/4057495611 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800cd80 con 0x7ff52c075b30 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c1aba10 con 0x7ff52c075b30 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff529ffb640 1 -- 172.21.15.107:0/4057495611 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5180108c0 con 0x7ff52c075b30 2024-08-20T06:39:40.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff529ffb640 1 -- 172.21.15.107:0/4057495611 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180187c0 con 0x7ff52c075b30 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.136+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c1abf70 con 0x7ff52c075b30 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.139+0000 7ff529ffb640 1 -- 172.21.15.107:0/4057495611 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff518010430 con 0x7ff52c075b30 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 Processor -- start 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 -- start start 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 0x55eef4aadc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 0x55eef4abc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef4a24930 con 0x55eef4aae1a0 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eef4a13e30 con 0x55eef4aba130 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff53564a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eef4a17e70 con 0x55eef4aad880 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff529ffb640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 0x7ff51c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff530fed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 0x55eef4abc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff5317ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 0x55eef4aadc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff530fed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41460/0 (socket says 172.21.15.107:41460) 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff530fed640 1 -- 172.21.15.107:0/3270892658 learned_addr learned my addr 172.21.15.107:0/3270892658 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff5317ee640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 0x7ff51c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.140+0000 7ff529ffb640 1 -- 172.21.15.107:0/4057495611 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff5180e3ed0 con 0x7ff52c075b30 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 -- 172.21.15.107:0/3270892658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 msgr2=0x55eef4abc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 0x55eef4abc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 -- 172.21.15.107:0/3270892658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 msgr2=0x55eef4ab9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 -- 172.21.15.107:0/3270892658 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef4773650 con 0x55eef4aad880 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff52bfff640 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 0x55eef4abc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:39:40.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff530fed640 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:40.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.141+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 0x55eef4aadc60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0761f0 tx=0x7ff52400f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809a955c471fd1c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.142+0000 7ff5317ee640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 0x7ff51c0b6110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff5240099e0 tx=0x7ff52401e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.142+0000 7ff516ffd640 1 -- 172.21.15.107:0/3270892658 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52402b070 con 0x55eef4aad880 2024-08-20T06:39:40.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.142+0000 7ff516ffd640 1 -- 172.21.15.107:0/3270892658 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5240268a0 con 0x55eef4aad880 2024-08-20T06:39:40.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.142+0000 7ff516ffd640 1 -- 172.21.15.107:0/3270892658 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff524022070 con 0x55eef4aad880 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- 172.21.15.107:0/3270892658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 msgr2=0x55eef4aadc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 0x55eef4aadc60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff52c0761f0 tx=0x7ff52400f950 comp rx=0 tx=0).stop 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- 172.21.15.107:0/3270892658 shutdown_connections 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aba130 0x55eef4abc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aae1a0 0x55eef4ab9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 --2- 172.21.15.107:0/3270892658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aad880 0x55eef4aadc60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- 172.21.15.107:0/3270892658 >> 172.21.15.107:0/3270892658 conn(0x55eef4aab6b0 msgr2=0x55eef4aac850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- 172.21.15.107:0/3270892658 shutdown_connections 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- 172.21.15.107:0/3270892658 wait complete. 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 -- start start 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.143+0000 7ff53564a740 1 Processor -- start 2024-08-20T06:39:40.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4ab7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4ab75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4ab7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55eef4876700 con 0x55eef4aad880 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4ab75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55eef499a810 con 0x55eef4aae1a0 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4ab75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43796/0 (socket says 172.21.15.107:43796) 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff52bfff640 1 -- 172.21.15.107:0/3599409188 learned_addr learned my addr 172.21.15.107:0/3599409188 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55eef4abcd80 con 0x55eef4aba130 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff530fed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4ab7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4ab7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.144+0000 7ff53564a740 10 client.0 _ll_register_callbacks cb 0x55eef4a0d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:40.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 -- 172.21.15.107:0/3599409188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 msgr2=0x55eef4ab75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4ab75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 -- 172.21.15.107:0/3599409188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 msgr2=0x55eef4ab7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4ab7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 -- 172.21.15.107:0/3599409188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef4ac6100 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff530fed640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4ab7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4ab7b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff524000f80 tx=0x7ff524004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2386126ae02e7a05 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff52402b040 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff524005040 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eef49c3150 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.145+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff524022040 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 10 client.25225 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eef4ac89e0 con 0x55eef4aba130 2024-08-20T06:39:40.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff524034070 con 0x55eef4aba130 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff52402e070 con 0x55eef4aba130 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 10 client.25225 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 10 client.25225 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 20 client.25225 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4ac9ea0 0x55eef4acc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 4 client.25225 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eef49c3150 con 0x55eef4ac9ea0 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.146+0000 7ff530fed640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4ac9ea0 0x55eef4acc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.147+0000 7ff530fed640 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4ac9ea0 0x55eef4acc290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1faf3915d31a6e8 server_cookie=81222a7a4f9f0544 in_seq=0 out_seq=0 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.147+0000 7ff50bfff640 10 client.25225 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.147+0000 7ff50bfff640 1 -- 172.21.15.107:0/3599409188 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eef49c3150 con 0x55eef4ac9ea0 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.147+0000 7ff50bfff640 10 client.25225 handle_command_reply: tid=0 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.147+0000 7ff53564a740 1 client.25225 shutdown 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 msgr2=0x55eef4ab7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4ab7b10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff524000f80 tx=0x7ff524004ef0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 shutdown_connections 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4ac9ea0 0x55eef4acc290 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff50bfff640 0 client.25225 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4ab7b10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4ab75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 --2- 172.21.15.107:0/3599409188 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4ab7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 >> 172.21.15.107:0/3599409188 conn(0x55eef4aab6b0 msgr2=0x55eef4ab1250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 shutdown_connections 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 1 -- 172.21.15.107:0/3599409188 wait complete. 2024-08-20T06:39:40.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.148+0000 7ff53564a740 20 client.25225 trim_cache size 0 max 16384 2024-08-20T06:39:40.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.166+0000 7ff53564a740 1 Processor -- start 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.166+0000 7ff53564a740 1 -- start start 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aae1a0 0x55eef4ad1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aba130 0x55eef4ad1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b6f80 con 0x55eef4aba130 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43810/0 (socket says 172.21.15.107:43810) 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 -- 172.21.15.107:0/307603548 learned_addr learned my addr 172.21.15.107:0/307603548 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b4850 con 0x55eef4aad880 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b2680 con 0x55eef4aae1a0 2024-08-20T06:39:40.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff5317ee640 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aba130 0x55eef4ad1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff52bfff640 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aae1a0 0x55eef4ad1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 -- 172.21.15.107:0/307603548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aae1a0 msgr2=0x55eef4ad1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aae1a0 0x55eef4ad1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 -- 172.21.15.107:0/307603548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aba130 msgr2=0x55eef4ad1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aba130 0x55eef4ad1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.167+0000 7ff530fed640 1 -- 172.21.15.107:0/307603548 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eef4aac130 con 0x55eef4aad880 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.168+0000 7ff530fed640 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5200057c0 tx=0x7ff5200040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e02be9d345622e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.168+0000 7ff5167fc640 1 -- 172.21.15.107:0/307603548 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520035040 con 0x55eef4aad880 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.168+0000 7ff5167fc640 1 -- 172.21.15.107:0/307603548 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff520003a80 con 0x55eef4aad880 2024-08-20T06:39:40.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.168+0000 7ff5167fc640 1 -- 172.21.15.107:0/307603548 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff520032580 con 0x55eef4aad880 2024-08-20T06:39:40.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 msgr2=0x55eef4ab8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff5200057c0 tx=0x7ff5200040f0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 shutdown_connections 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aba130 0x55eef4ad1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aae1a0 0x55eef4ad1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 --2- 172.21.15.107:0/307603548 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aad880 0x55eef4ab8dd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 >> 172.21.15.107:0/307603548 conn(0x55eef4aab6b0 msgr2=0x55eef4ab86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 shutdown_connections 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- 172.21.15.107:0/307603548 wait complete. 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 -- start start 2024-08-20T06:39:40.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.169+0000 7ff53564a740 1 Processor -- start 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4abf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4b32a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b2d10 con 0x55eef4aad880 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b4180 con 0x55eef4aae1a0 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.170+0000 7ff53564a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c1b33e0 con 0x55eef4aba130 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff53564a740 10 client.0 _ll_register_callbacks cb 0x7ff524029070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43826/0 (socket says 172.21.15.107:43826) 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 -- 172.21.15.107:0/3089847045 learned_addr learned my addr 172.21.15.107:0/3089847045 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:40.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff5317ee640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4b32a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff530fed640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4abf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 -- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 msgr2=0x55eef4b32a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4b32a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 -- 172.21.15.107:0/3089847045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 msgr2=0x55eef4abf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4abf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 -- 172.21.15.107:0/3089847045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eef4b32fb0 con 0x55eef4aae1a0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff52bfff640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a8140 tx=0x7ff518074400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34a94f03dc9883e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff50bfff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5180b3c40 con 0x55eef4aae1a0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff5180740a0 con 0x55eef4aae1a0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.171+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff520003a80 con 0x55eef4aae1a0 2024-08-20T06:39:40.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff5180c5050 con 0x55eef4aae1a0 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 10 client.25322 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55eef4b34120 con 0x55eef4aae1a0 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5180aaed0 con 0x55eef4aae1a0 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff5180a6840 con 0x55eef4aae1a0 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 10 client.25322 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 10 client.25322 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 20 client.25322 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4b34920 0x55eef4b36d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:40.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 4 client.25322 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:39:40.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.172+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff520003a80 con 0x55eef4b34920 2024-08-20T06:39:40.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff530fed640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4b34920 0x55eef4b36d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff50bfff640 10 client.25322 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff530fed640 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4b34920 0x55eef4b36d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe1bd0eb2ff6bd5d server_cookie=73ec2c4fdcf37e47 in_seq=0 out_seq=0 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff50bfff640 1 -- 172.21.15.107:0/3089847045 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff520003a80 con 0x55eef4b34920 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff50bfff640 10 client.25322 handle_command_reply: tid=0 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 client.25322 shutdown 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 msgr2=0x55eef4abf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff5180a8140 tx=0x7ff518074400 comp rx=0 tx=0).stop 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 shutdown_connections 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55eef4b34920 0x55eef4b36d40 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eef4aba130 0x55eef4b32a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55eef4aae1a0 0x55eef4abf720 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 --2- 172.21.15.107:0/3089847045 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55eef4aad880 0x55eef4abf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 >> 172.21.15.107:0/3089847045 conn(0x55eef4aab6b0 msgr2=0x55eef4ab8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.173+0000 7ff50bfff640 0 client.25322 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.174+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 shutdown_connections 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.174+0000 7ff53564a740 1 -- 172.21.15.107:0/3089847045 wait complete. 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.174+0000 7ff53564a740 20 client.25322 trim_cache size 0 max 16384 2024-08-20T06:39:40.180 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 msgr2=0x7ff51c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 0x7ff51c0b6110 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff5240099e0 tx=0x7ff52401e000 comp rx=0 tx=0).stop 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 msgr2=0x7ff52c10bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff51800e9f0 tx=0x7ff51800eeb0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 shutdown_connections 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff52c10f090 0x7ff52c10c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.181 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff52c075b30 0x7ff52c10bbd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff51c0b3c50 0x7ff51c0b6110 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 --2- 172.21.15.107:0/4057495611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff52c075210 0x7ff52c10e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 >> 172.21.15.107:0/4057495611 conn(0x7ff52c0fe550 msgr2=0x7ff52c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 shutdown_connections 2024-08-20T06:39:40.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:40.175+0000 7ff533251640 1 -- 172.21.15.107:0/4057495611 wait complete. 2024-08-20T06:39:41.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:39:41.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:41 smithi107 ceph-mon[18495]: pgmap v412: 97 pgs: 97 active+clean; 134 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-08-20T06:39:41.374 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:41.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:41 smithi183 ceph-mon[22204]: pgmap v412: 97 pgs: 97 active+clean; 134 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-08-20T06:39:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:41 smithi193 ceph-mon[21944]: pgmap v412: 97 pgs: 97 active+clean; 134 MiB data, 840 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.906+0000 7fe47a2a0640 1 -- 172.21.15.107:0/1272764718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 msgr2=0x7fe4741077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.906+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/1272764718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe4741077e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe468009900 tx=0x7fe46802a410 comp rx=0 tx=0).stop 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 -- 172.21.15.107:0/1272764718 shutdown_connections 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/1272764718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe47410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/1272764718 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe4741077e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/1272764718 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 0x7fe474101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 -- 172.21.15.107:0/1272764718 >> 172.21.15.107:0/1272764718 conn(0x7fe4740fd400 msgr2=0x7fe4740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 -- 172.21.15.107:0/1272764718 shutdown_connections 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.907+0000 7fe47a2a0640 1 -- 172.21.15.107:0/1272764718 wait complete. 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 Processor -- start 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 -- start start 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 0x7fe47419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe4741a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe474111850 con 0x7fe4741073a0 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741116d0 con 0x7fe474101990 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe47a2a0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741119d0 con 0x7fe474107d20 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.908+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe473fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 0x7fe47419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41512/0 (socket says 172.21.15.107:41512) 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 -- 172.21.15.107:0/2871767217 learned_addr learned my addr 172.21.15.107:0/2871767217 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:41.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe47883d640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe4741a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 -- 172.21.15.107:0/2871767217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 msgr2=0x7fe4741a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe4741a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 -- 172.21.15.107:0/2871767217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 msgr2=0x7fe47419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 0x7fe47419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 -- 172.21.15.107:0/2871767217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4741a5570 con 0x7fe4741073a0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe47883d640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe4741a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.909+0000 7fe4737fe640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe468009230 tx=0x7fe4680319a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.910+0000 7fe4717fa640 1 -- 172.21.15.107:0/2871767217 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46803f030 con 0x7fe4741073a0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.910+0000 7fe4717fa640 1 -- 172.21.15.107:0/2871767217 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4680043d0 con 0x7fe4741073a0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.910+0000 7fe4717fa640 1 -- 172.21.15.107:0/2871767217 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468035070 con 0x7fe4741073a0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.910+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4741a5800 con 0x7fe4741073a0 2024-08-20T06:39:41.913 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.910+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4741a5bd0 con 0x7fe4741073a0 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.912+0000 7fe4717fa640 1 -- 172.21.15.107:0/2871767217 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fe468042030 con 0x7fe4741073a0 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 Processor -- start 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 -- start start 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc94c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbca1130 0x5572dbca3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5572dbc0b930 con 0x5572dbc94880 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572dbbfae30 con 0x5572dbc951a0 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47c699740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572dbbfee70 con 0x5572dbca1130 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe473fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe473fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43870/0 (socket says 172.21.15.107:43870) 2024-08-20T06:39:41.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe473fff640 1 -- 172.21.15.107:0/3439757092 learned_addr learned my addr 172.21.15.107:0/3439757092 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe47883d640 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc94c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.913+0000 7fe4737fe640 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbca1130 0x5572dbca3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 -- 172.21.15.107:0/3439757092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbca1130 msgr2=0x5572dbca3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbca1130 0x5572dbca3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 -- 172.21.15.107:0/3439757092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 msgr2=0x5572dbc94c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc94c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 -- 172.21.15.107:0/3439757092 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572db95a650 con 0x5572dbc951a0 2024-08-20T06:39:41.916 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe473fff640 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0114f0 tx=0x7fe46c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b83781cba79fa72 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe4717fa640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 0x7fe45c0b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe45a7fc640 1 -- 172.21.15.107:0/3439757092 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c01e070 con 0x5572dbc951a0 2024-08-20T06:39:41.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe45a7fc640 1 -- 172.21.15.107:0/3439757092 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe46c00fab0 con 0x5572dbc951a0 2024-08-20T06:39:41.919 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe45a7fc640 1 -- 172.21.15.107:0/3439757092 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c019aa0 con 0x5572dbc951a0 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.914+0000 7fe47883d640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 0x7fe45c0b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe4717fa640 1 -- 172.21.15.107:0/2871767217 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe468103eb0 con 0x7fe4741073a0 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 -- 172.21.15.107:0/3439757092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 msgr2=0x5572dbca0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0114f0 tx=0x7fe46c0119b0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47883d640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 0x7fe45c0b6260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe464012640 tx=0x7fe464005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 -- 172.21.15.107:0/3439757092 shutdown_connections 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbca1130 0x5572dbca3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbc951a0 0x5572dbca0bf0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 --2- 172.21.15.107:0/3439757092 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc94c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 -- 172.21.15.107:0/3439757092 >> 172.21.15.107:0/3439757092 conn(0x5572dbc926b0 msgr2=0x5572dbc93850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 -- 172.21.15.107:0/3439757092 shutdown_connections 2024-08-20T06:39:41.920 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.915+0000 7fe47c699740 1 -- 172.21.15.107:0/3439757092 wait complete. 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 -- start start 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 Processor -- start 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbca0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5572dba5d700 con 0x5572dbc94880 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5572dbb81810 con 0x5572dbca1130 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5572dbca3e80 con 0x5572dbc951a0 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.916+0000 7fe47c699740 10 client.0 _ll_register_callbacks cb 0x5572dbbf4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe47883d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe47883d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43884/0 (socket says 172.21.15.107:43884) 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe47883d640 1 -- 172.21.15.107:0/1039539491 learned_addr learned my addr 172.21.15.107:0/1039539491 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.921 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe473fff640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbca0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 -- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 msgr2=0x5572dbc9e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 -- 172.21.15.107:0/1039539491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 msgr2=0x5572dbca0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbca0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 -- 172.21.15.107:0/1039539491 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dbcad1b0 con 0x5572dbc951a0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9dff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5572dbc94830 tx=0x7fe4680cad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e75de633edfa492 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.917+0000 7fe4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468049040 con 0x5572dbc951a0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5572dbbaa150 con 0x5572dbc951a0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe4680cb2e0 con 0x5572dbc951a0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe4680ca980 con 0x5572dbc951a0 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe47c699740 10 client.25243 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:41.922 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572dbcafb10 con 0x5572dbc951a0 2024-08-20T06:39:41.923 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.918+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4680cae70 con 0x5572dbc951a0 2024-08-20T06:39:41.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe4680c67e0 con 0x5572dbc951a0 2024-08-20T06:39:41.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 10 client.25243 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:41.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 10 client.25243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 20 client.25243 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbcb1130 0x5572dbcb3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 4 client.25243 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572dbbaa150 con 0x5572dbcb1130 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.919+0000 7fe473fff640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbcb1130 0x5572dbcb3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.920+0000 7fe4537fe640 10 client.25243 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.920+0000 7fe473fff640 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbcb1130 0x5572dbcb3590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26b4fb19cccf4cdf server_cookie=6370316527b6a470 in_seq=0 out_seq=0 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.922+0000 7fe4537fe640 1 -- 172.21.15.107:0/1039539491 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572dbbaa150 con 0x5572dbcb1130 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.922+0000 7fe4537fe640 10 client.25243 handle_command_reply: tid=0 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.922+0000 7fe47c699740 1 client.25243 shutdown 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 msgr2=0x5572dbc9dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9dff0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x5572dbc94830 tx=0x7fe4680cad00 comp rx=0 tx=0).stop 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 shutdown_connections 2024-08-20T06:39:41.925 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbcb1130 0x5572dbcb3590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9dff0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 --2- 172.21.15.107:0/1039539491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbca0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 >> 172.21.15.107:0/1039539491 conn(0x5572dbc926b0 msgr2=0x5572dbc98250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe4537fe640 0 client.25243 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 shutdown_connections 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 1 -- 172.21.15.107:0/1039539491 wait complete. 2024-08-20T06:39:41.926 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.923+0000 7fe47c699740 20 client.25243 trim_cache size 0 max 16384 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 Processor -- start 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 -- start start 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc98a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b6e20 con 0x5572dbc94880 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b46f0 con 0x5572dbca1130 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47c699740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b2520 con 0x5572dbc951a0 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe473fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc98a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe473fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc98a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41546/0 (socket says 172.21.15.107:41546) 2024-08-20T06:39:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe473fff640 1 -- 172.21.15.107:0/1069197566 learned_addr learned my addr 172.21.15.107:0/1069197566 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.941+0000 7fe47883d640 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 -- 172.21.15.107:0/1069197566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 msgr2=0x5572dbc9c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 -- 172.21.15.107:0/1069197566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 msgr2=0x5572dbc98a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc98a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 -- 172.21.15.107:0/1069197566 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572dbc9c4f0 con 0x5572dbc951a0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9df60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4680c80e0 tx=0x7fe4680cad70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bfb2bb29df89fb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe459ffb640 1 -- 172.21.15.107:0/1069197566 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468049040 con 0x5572dbc951a0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe459ffb640 1 -- 172.21.15.107:0/1069197566 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe4680cf3c0 con 0x5572dbc951a0 2024-08-20T06:39:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.942+0000 7fe459ffb640 1 -- 172.21.15.107:0/1069197566 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4680d39e0 con 0x5572dbc951a0 2024-08-20T06:39:41.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- 172.21.15.107:0/1069197566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 msgr2=0x5572dbc9df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9df60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe4680c80e0 tx=0x7fe4680cad70 comp rx=0 tx=0).stop 2024-08-20T06:39:41.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- 172.21.15.107:0/1069197566 shutdown_connections 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbc9c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc951a0 0x5572dbc9df60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 --2- 172.21.15.107:0/1069197566 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc94880 0x5572dbc98a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- 172.21.15.107:0/1069197566 >> 172.21.15.107:0/1069197566 conn(0x5572dbc926b0 msgr2=0x5572dbc992d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- 172.21.15.107:0/1069197566 shutdown_connections 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- 172.21.15.107:0/1069197566 wait complete. 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.943+0000 7fe47c699740 1 -- start start 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 Processor -- start 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc951a0 0x5572dbca40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbca4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b2bb0 con 0x5572dbc951a0 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b4020 con 0x5572dbca1130 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe4741b3280 con 0x5572dbc94880 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe4737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc951a0 0x5572dbca40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:44580/0 (socket says 172.21.15.107:44580) 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47c699740 10 client.0 _ll_register_callbacks cb 0x7fe4680c6d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 -- 172.21.15.107:0/1699780730 learned_addr learned my addr 172.21.15.107:0/1699780730 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe47883d640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbca4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 -- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 msgr2=0x5572dbca4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbca4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 -- 172.21.15.107:0/1699780730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc951a0 msgr2=0x5572dbca40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc951a0 0x5572dbca40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.944+0000 7fe473fff640 1 -- 172.21.15.107:0/1699780730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572dbd19e60 con 0x5572dbc94880 2024-08-20T06:39:41.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe473fff640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0195d0 tx=0x7fe46c009b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=262bf11b05152d42 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe4537fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c01e040 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4680cf3c0 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe46c003970 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe46c003b70 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe47c699740 10 client.25249 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5572dbd1b030 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.945+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe46c029070 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fe46c00f620 con 0x5572dbc94880 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 10 client.25249 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 10 client.25249 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 20 client.25249 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbd1b6f0 0x5572dbd1db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:41.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 4 client.25249 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fe4680cf3c0 con 0x5572dbd1b6f0 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.946+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbd1b6f0 0x5572dbd1db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe4537fe640 10 client.25249 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe4737fe640 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbd1b6f0 0x5572dbd1db10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d425effebf7eaa server_cookie=576ac074e558e116 in_seq=0 out_seq=0 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe4537fe640 1 -- 172.21.15.107:0/1699780730 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fe4680cf3c0 con 0x5572dbd1b6f0 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe4537fe640 10 client.25249 handle_command_reply: tid=0 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 client.25249 shutdown 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 msgr2=0x5572dbca9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe46c0195d0 tx=0x7fe46c009b60 comp rx=0 tx=0).stop 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 shutdown_connections 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5572dbca1130 0x5572dbca4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5572dbc951a0 0x5572dbca40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5572dbd1b6f0 0x5572dbd1db10 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 --2- 172.21.15.107:0/1699780730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5572dbc94880 0x5572dbca9920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe4537fe640 0 client.25249 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 >> 172.21.15.107:0/1699780730 conn(0x5572dbc926b0 msgr2=0x5572dbc9f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 shutdown_connections 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.947+0000 7fe47c699740 1 -- 172.21.15.107:0/1699780730 wait complete. 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.948+0000 7fe47c699740 20 client.25249 trim_cache size 0 max 16384 2024-08-20T06:39:41.954 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:39:41.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.948+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 msgr2=0x7fe45c0b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.948+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 0x7fe45c0b6260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe464012640 tx=0x7fe464005f90 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.948+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 msgr2=0x7fe47419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.948+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe468009230 tx=0x7fe4680319a0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 shutdown_connections 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe45c0b3da0 0x7fe45c0b6260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe474107d20 0x7fe4741a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe4741073a0 0x7fe47419fbe0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 --2- 172.21.15.107:0/2871767217 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe474101990 0x7fe47419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 >> 172.21.15.107:0/2871767217 conn(0x7fe4740fd400 msgr2=0x7fe4740ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 shutdown_connections 2024-08-20T06:39:41.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:41.949+0000 7fe47a2a0640 1 -- 172.21.15.107:0/2871767217 wait complete. 2024-08-20T06:39:43.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:43 smithi107 ceph-mon[18495]: pgmap v413: 97 pgs: 97 active+clean; 135 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-08-20T06:39:43.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:43 smithi183 ceph-mon[22204]: pgmap v413: 97 pgs: 97 active+clean; 135 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-08-20T06:39:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:43 smithi193 ceph-mon[21944]: pgmap v413: 97 pgs: 97 active+clean; 135 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 1.6 MiB/s wr, 115 op/s 2024-08-20T06:39:43.901 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:39:43.901 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:39:44.245 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2399371851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b481073a0 msgr2=0x7f3b481077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2399371851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b481073a0 0x7f3b481077e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009900 tx=0x7f3b3c02a3f0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2399371851 shutdown_connections 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2399371851 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48107d20 0x7f3b4810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2399371851 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b481073a0 0x7f3b481077e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2399371851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48101990 0x7f3b48101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2399371851 >> 172.21.15.107:0/2399371851 conn(0x7f3b480fd400 msgr2=0x7f3b480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.778+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2399371851 shutdown_connections 2024-08-20T06:39:44.780 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.779+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2399371851 wait complete. 2024-08-20T06:39:44.781 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.779+0000 7f3b4e14b640 1 Processor -- start 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.779+0000 7f3b4e14b640 1 -- start start 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 0x7f3b48198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b481073a0 0x7f3b48199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48111820 con 0x7f3b48107d20 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481116a0 con 0x7f3b48101990 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b4e14b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481119a0 con 0x7f3b481073a0 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.782 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:41560/0 (socket says 172.21.15.107:41560) 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 -- 172.21.15.107:0/2590871227 learned_addr learned my addr 172.21.15.107:0/2590871227 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 0x7f3b48198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b481073a0 0x7f3b48199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 -- 172.21.15.107:0/2590871227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b481073a0 msgr2=0x7f3b48199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b481073a0 0x7f3b48199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 -- 172.21.15.107:0/2590871227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 msgr2=0x7f3b48198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 0x7f3b48198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 -- 172.21.15.107:0/2590871227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4819ef70 con 0x7f3b48107d20 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b477fe640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 0x7f3b48198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.780+0000 7f3b47fff640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000eb10 tx=0x7f3b4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.781+0000 7f3b44ff9640 1 -- 172.21.15.107:0/2590871227 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40004b20 con 0x7f3b48107d20 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.781+0000 7f3b44ff9640 1 -- 172.21.15.107:0/2590871227 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b40004cc0 con 0x7f3b48107d20 2024-08-20T06:39:44.783 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.781+0000 7f3b44ff9640 1 -- 172.21.15.107:0/2590871227 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40010690 con 0x7f3b48107d20 2024-08-20T06:39:44.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.781+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4819f260 con 0x7f3b48107d20 2024-08-20T06:39:44.784 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.781+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481084f0 con 0x7f3b48107d20 2024-08-20T06:39:44.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 Processor -- start 2024-08-20T06:39:44.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b44ff9640 1 -- 172.21.15.107:0/2590871227 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f3b40005020 con 0x7f3b48107d20 2024-08-20T06:39:44.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 -- start start 2024-08-20T06:39:44.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.785 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e07720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9d71940 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9d60e40 con 0x55c0b9e07c60 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.784+0000 7f3b50544740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9d64e80 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b46ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b46ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:44614/0 (socket says 172.21.15.107:44614) 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b46ffd640 1 -- 172.21.15.107:0/671268492 learned_addr learned my addr 172.21.15.107:0/671268492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e07720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.786 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 -- 172.21.15.107:0/671268492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 msgr2=0x55c0b9dfb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 -- 172.21.15.107:0/671268492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9e07c60 msgr2=0x55c0b9e0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 -- 172.21.15.107:0/671268492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b9cfd520 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.787 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b44ff9640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 0x7f3b300b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.785+0000 7f3b477fe640 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e07720 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3b38010400 tx=0x7f3b380108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cc0ce23996b7a1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.786+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 0x7f3b300b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.786+0000 7f3b44ff9640 1 -- 172.21.15.107:0/2590871227 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3b400e41f0 con 0x7f3b48107d20 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.786+0000 7f3b2dffb640 1 -- 172.21.15.107:0/671268492 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38004220 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.786+0000 7f3b2dffb640 1 -- 172.21.15.107:0/671268492 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b380043c0 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.788 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.786+0000 7f3b2dffb640 1 -- 172.21.15.107:0/671268492 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38003970 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- 172.21.15.107:0/671268492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 msgr2=0x55c0b9e07720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 0x7f3b300b6190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55c0b9dfb360 tx=0x7f3b3c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e07720 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3b38010400 tx=0x7f3b380108c0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- 172.21.15.107:0/671268492 shutdown_connections 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e07720 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 --2- 172.21.15.107:0/671268492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dfb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- 172.21.15.107:0/671268492 >> 172.21.15.107:0/671268492 conn(0x55c0b9df91e0 msgr2=0x55c0b9dfa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- 172.21.15.107:0/671268492 shutdown_connections 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- 172.21.15.107:0/671268492 wait complete. 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.787+0000 7f3b50544740 1 -- start start 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 Processor -- start 2024-08-20T06:39:44.791 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e04bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e05670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e04bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e04bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43924/0 (socket says 172.21.15.107:43924) 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b477fe640 1 -- 172.21.15.107:0/4102162186 learned_addr learned my addr 172.21.15.107:0/4102162186 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9bc3890 con 0x55c0b9e07c60 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e05670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9ce7700 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.788+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c0b9e0a9b0 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b50544740 10 client.0 _ll_register_callbacks cb 0x55c0b9d5c6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 -- 172.21.15.107:0/4102162186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfbcd0 msgr2=0x55c0b9e05130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.792 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 -- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 msgr2=0x55c0b9e04bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e04bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b9ac0930 con 0x55c0b9e07c60 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e05670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3b400e1c80 tx=0x7f3b400aab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af61c11b381563e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400c4020 con 0x55c0b9e07c60 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.789+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b380043c0 con 0x55c0b9e07c60 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3b400ab570 con 0x55c0b9e07c60 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b400e3040 con 0x55c0b9e07c60 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b50544740 10 client.15834 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:44.793 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0b9e16600 con 0x55c0b9e07c60 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b380043c0 con 0x55c0b9e07c60 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b400b3d20 con 0x55c0b9e07c60 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b50544740 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.790+0000 7f3b50544740 10 client.15834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b50544740 20 client.15834 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e17f10 0x55c0b9e1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b50544740 4 client.15834 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0b9d73890 con 0x55c0b9e17f10 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e17f10 0x55c0b9e1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e17f10 0x55c0b9e1a3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e77dbce4420a682 server_cookie=89eb99deedd9f07a in_seq=0 out_seq=0 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.791+0000 7f3b26ffd640 10 client.15834 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4102162186 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0b9d73890 con 0x55c0b9e17f10 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b26ffd640 10 client.15834 handle_command_reply: tid=0 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 client.15834 shutdown 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 msgr2=0x55c0b9e05670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e05670 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3b400e1c80 tx=0x7f3b400aab40 comp rx=0 tx=0).stop 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 shutdown_connections 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e05670 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e17f10 0x55c0b9e1a3b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 --2- 172.21.15.107:0/4102162186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e04bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 >> 172.21.15.107:0/4102162186 conn(0x55c0b9df91e0 msgr2=0x55c0b9dfed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b26ffd640 0 client.15834 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:44.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 shutdown_connections 2024-08-20T06:39:44.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 1 -- 172.21.15.107:0/4102162186 wait complete. 2024-08-20T06:39:44.797 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.792+0000 7f3b50544740 20 client.15834 trim_cache size 0 max 16384 2024-08-20T06:39:44.821 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 Processor -- start 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 -- start start 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 0x55c0b9e1fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.819+0000 7f3b50544740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481b2b30 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b50544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481b0400 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b50544740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481ae230 con 0x55c0b9e07c60 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43940/0 (socket says 172.21.15.107:43940) 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 -- 172.21.15.107:0/341721923 learned_addr learned my addr 172.21.15.107:0/341721923 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 -- 172.21.15.107:0/341721923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 msgr2=0x55c0b9e1fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b47fff640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 0x55c0b9e1fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 0x55c0b9e1fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 -- 172.21.15.107:0/341721923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 msgr2=0x55c0b9e05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 -- 172.21.15.107:0/341721923 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0b9e0a9b0 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b47fff640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 0x55c0b9e1fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:39:44.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b477fe640 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800bf00 tx=0x7f3b380051d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d8b4d0e12f03ad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.820+0000 7f3b2d7fa640 1 -- 172.21.15.107:0/341721923 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38035070 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.821+0000 7f3b2d7fa640 1 -- 172.21.15.107:0/341721923 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b38005540 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.821+0000 7f3b2d7fa640 1 -- 172.21.15.107:0/341721923 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b380048e0 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.821+0000 7f3b50544740 1 -- 172.21.15.107:0/341721923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 msgr2=0x55c0b9dff420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.821+0000 7f3b50544740 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800bf00 tx=0x7f3b380051d0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 -- 172.21.15.107:0/341721923 shutdown_connections 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9e07c60 0x55c0b9e1fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 --2- 172.21.15.107:0/341721923 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9dff420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 -- 172.21.15.107:0/341721923 >> 172.21.15.107:0/341721923 conn(0x55c0b9df91e0 msgr2=0x55c0b9e04b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 -- 172.21.15.107:0/341721923 shutdown_connections 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 -- 172.21.15.107:0/341721923 wait complete. 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 -- start start 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.822+0000 7f3b50544740 1 Processor -- start 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481ae8c0 con 0x55c0b9e07c60 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481afd30 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481aef90 con 0x55c0b9dfb3b0 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b50544740 10 client.0 _ll_register_callbacks cb 0x7f3b40004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43956/0 (socket says 172.21.15.107:43956) 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 -- 172.21.15.107:0/4142653660 learned_addr learned my addr 172.21.15.107:0/4142653660 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 -- 172.21.15.107:0/4142653660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 msgr2=0x55c0b9e104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 -- 172.21.15.107:0/4142653660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 msgr2=0x55c0b9e0c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b46ffd640 1 -- 172.21.15.107:0/4142653660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0b9e8a4a0 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b47fff640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.823+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:44.830 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b46ffd640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0057c0 tx=0x7f3b3c0056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4f5c9c81574597 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3c058060 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3b3c003bd0 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b38005540 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b3c035040 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b50544740 10 client.25358 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c0b9e8b690 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.824+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3b3c003dd0 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3b3c047040 con 0x55c0b9dfbcd0 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 10 client.25358 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 10 client.25358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 20 client.25358 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:44.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e8be40 0x55c0b9e8e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 4 client.25358 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f3b38005540 con 0x55c0b9e8be40 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e8be40 0x55c0b9e8e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b477fe640 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e8be40 0x55c0b9e8e260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73b767124cc401ca server_cookie=4d39bb7adda1785e in_seq=0 out_seq=0 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.825+0000 7f3b26ffd640 10 client.25358 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b26ffd640 1 -- 172.21.15.107:0/4142653660 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f3b38005540 con 0x55c0b9e8be40 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b26ffd640 10 client.25358 handle_command_reply: tid=0 2024-08-20T06:39:44.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 client.25358 shutdown 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 msgr2=0x55c0b9e0b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c0057c0 tx=0x7f3b3c0056d0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 shutdown_connections 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c0b9e07c60 0x55c0b9e0c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c0b9dfbcd0 0x55c0b9e0b9c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55c0b9e8be40 0x55c0b9e8e260 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 --2- 172.21.15.107:0/4142653660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c0b9dfb3b0 0x55c0b9e104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 >> 172.21.15.107:0/4142653660 conn(0x55c0b9df91e0 msgr2=0x55c0b9e05380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b26ffd640 0 client.25358 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 shutdown_connections 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 1 -- 172.21.15.107:0/4142653660 wait complete. 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.827+0000 7f3b50544740 20 client.25358 trim_cache size 0 max 16384 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:39:44.834 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "d9d50c8e-8c2c-4a6b-bb1e-a883122b1f0e", 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 msgr2=0x7f3b300b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 0x7f3b300b6190 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55c0b9dfb360 tx=0x7f3b3c009260 comp rx=0 tx=0).stop 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 msgr2=0x7f3b4819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000eb10 tx=0x7f3b4000c2d0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 shutdown_connections 2024-08-20T06:39:44.835 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3b48107d20 0x7f3b4819e7a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f3b300b3cd0 0x7f3b300b6190 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b481073a0 0x7f3b48199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 --2- 172.21.15.107:0/2590871227 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3b48101990 0x7f3b48198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 >> 172.21.15.107:0/2590871227 conn(0x7f3b480fd400 msgr2=0x7f3b48104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 shutdown_connections 2024-08-20T06:39:44.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:44.828+0000 7f3b4e14b640 1 -- 172.21.15.107:0/2590871227 wait complete. 2024-08-20T06:39:45.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:45 smithi107 ceph-mon[18495]: pgmap v414: 97 pgs: 97 active+clean; 140 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-08-20T06:39:45.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:45 smithi183 ceph-mon[22204]: pgmap v414: 97 pgs: 97 active+clean; 140 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-08-20T06:39:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:45 smithi193 ceph-mon[21944]: pgmap v414: 97 pgs: 97 active+clean; 140 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-08-20T06:39:45.696 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:39:45.885 INFO:tasks.workunit.client.0.smithi107.stdout: 35 15319 247 9697 128 2417 56 2024-08-20T06:39:46.063 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:39:46.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:39:46.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:39:46.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:39:46.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:39:46.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:39:46.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:46 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:39:46.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:46 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:39:46.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:46 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 -- 172.21.15.107:0/2758884311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 msgr2=0x7f162c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.588 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 --2- 172.21.15.107:0/2758884311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c10f250 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1624009ca0 tx=0x7f162402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 -- 172.21.15.107:0/2758884311 shutdown_connections 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 --2- 172.21.15.107:0/2758884311 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c10f250 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 --2- 172.21.15.107:0/2758884311 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c10c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 --2- 172.21.15.107:0/2758884311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 -- 172.21.15.107:0/2758884311 >> 172.21.15.107:0/2758884311 conn(0x7f162c0fd400 msgr2=0x7f162c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 -- 172.21.15.107:0/2758884311 shutdown_connections 2024-08-20T06:39:46.589 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.587+0000 7f16345ad640 1 -- 172.21.15.107:0/2758884311 wait complete. 2024-08-20T06:39:46.591 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f16345ad640 1 Processor -- start 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f16345ad640 1 -- start start 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f16345ad640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c19b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f16345ad640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c19b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f16345ad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f1632b4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f1632349640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c19b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f1632b4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58246/0 (socket says 172.21.15.107:58246) 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.588+0000 7f1632349640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c19b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:43590/0 (socket says 172.21.15.107:43590) 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 -- 172.21.15.107:0/3103460235 learned_addr learned my addr 172.21.15.107:0/3103460235 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f162c112910 con 0x7f162c10ce20 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f162c112790 con 0x7f162c104270 2024-08-20T06:39:46.592 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f162c112a90 con 0x7f162c103870 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1631b48640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c19b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 -- 172.21.15.107:0/3103460235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 msgr2=0x7f162c19b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c19b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 -- 172.21.15.107:0/3103460235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 msgr2=0x7f162c19b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c19b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f162c1a10e0 con 0x7f162c10ce20 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1631b48640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.589+0000 7f1632b4a640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1624031040 tx=0x7f1624003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.590+0000 7f16237fe640 1 -- 172.21.15.107:0/3103460235 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240307c0 con 0x7f162c10ce20 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.590+0000 7f16237fe640 1 -- 172.21.15.107:0/3103460235 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16240318e0 con 0x7f162c10ce20 2024-08-20T06:39:46.593 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.590+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f162c1a1370 con 0x7f162c10ce20 2024-08-20T06:39:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.590+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f162c1a7a90 con 0x7f162c10ce20 2024-08-20T06:39:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.590+0000 7f16237fe640 1 -- 172.21.15.107:0/3103460235 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1624004730 con 0x7f162c10ce20 2024-08-20T06:39:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.593+0000 7f16369a6740 1 Processor -- start 2024-08-20T06:39:46.594 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.593+0000 7f16237fe640 1 -- 172.21.15.107:0/3103460235 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f1624031450 con 0x7f162c10ce20 2024-08-20T06:39:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 -- start start 2024-08-20T06:39:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 0x556cfc849c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.595 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 0x556cfc855ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556cfc7c08e0 con 0x556cfc84a150 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556cfc7afde0 con 0x556cfc8560e0 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f16369a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556cfc7b3e20 con 0x556cfc849830 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f1632349640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 0x556cfc855ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f1632349640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 0x556cfc855ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58254/0 (socket says 172.21.15.107:58254) 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f1632349640 1 -- 172.21.15.107:0/4116368880 learned_addr learned my addr 172.21.15.107:0/4116368880 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:46.597 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f1631b48640 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 0x556cfc849c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.594+0000 7f1631b48640 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 -- 172.21.15.107:0/4116368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 msgr2=0x556cfc8584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 -- 172.21.15.107:0/4116368880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 msgr2=0x556cfc855ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 0x556cfc855ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 -- 172.21.15.107:0/4116368880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfc50f650 con 0x556cfc849830 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f16237fe640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 0x7f16180b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1631b48640 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 0x556cfc849c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f161c011ab0 tx=0x7f161c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f693ccc4e8a9743 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f16237fe640 1 -- 172.21.15.107:0/3103460235 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f1624104fa0 con 0x7f162c10ce20 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1632349640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 0x7f16180b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1620ff9640 1 -- 172.21.15.107:0/4116368880 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c004030 con 0x556cfc849830 2024-08-20T06:39:46.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1620ff9640 1 -- 172.21.15.107:0/4116368880 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f161c00f940 con 0x556cfc849830 2024-08-20T06:39:46.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.595+0000 7f1620ff9640 1 -- 172.21.15.107:0/4116368880 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c005360 con 0x556cfc849830 2024-08-20T06:39:46.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 -- 172.21.15.107:0/4116368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 msgr2=0x556cfc849c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f1632349640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 0x7f16180b6110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x556cfc84c9b0 tx=0x7f162800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 0x556cfc849c10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f161c011ab0 tx=0x7f161c004730 comp rx=0 tx=0).stop 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 -- 172.21.15.107:0/4116368880 shutdown_connections 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc84a150 0x556cfc855ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 --2- 172.21.15.107:0/4116368880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc849830 0x556cfc849c10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.596+0000 7f16369a6740 1 -- 172.21.15.107:0/4116368880 >> 172.21.15.107:0/4116368880 conn(0x556cfc847660 msgr2=0x556cfc848800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.597+0000 7f16369a6740 1 -- 172.21.15.107:0/4116368880 shutdown_connections 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.597+0000 7f16369a6740 1 -- 172.21.15.107:0/4116368880 wait complete. 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.597+0000 7f16369a6740 1 -- start start 2024-08-20T06:39:46.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f16369a6740 1 Processor -- start 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f16369a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8530a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f16369a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f16369a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc853b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f16369a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556cfc612700 con 0x556cfc849830 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f1631b48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f1632349640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f1631b48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:43612/0 (socket says 172.21.15.107:43612) 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f1631b48640 1 -- 172.21.15.107:0/1147506869 learned_addr learned my addr 172.21.15.107:0/1147506869 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.598+0000 7f1632b4a640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc853b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556cfc7367c0 con 0x556cfc8560e0 2024-08-20T06:39:46.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556cfc858d90 con 0x556cfc84a150 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f16369a6740 10 client.0 _ll_register_callbacks cb 0x556cfc7a96b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 -- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 msgr2=0x556cfc853b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc853b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 -- 172.21.15.107:0/1147506869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 msgr2=0x556cfc8530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfc86aa10 con 0x556cfc84a150 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556cfc75f100 con 0x556cfc84a150 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1631b48640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f161c0044c0 tx=0x7f161c0044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e624be7b55d8ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1615ffb640 10 client.25276 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1615ffb640 1 -- 172.21.15.107:0/1147506869 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c00fb50 con 0x556cfc84a150 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1615ffb640 1 -- 172.21.15.107:0/1147506869 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f161c00fcf0 con 0x556cfc84a150 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.599+0000 7f1615ffb640 1 -- 172.21.15.107:0/1147506869 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f161c005020 con 0x556cfc84a150 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 10 client.25276 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=27}) -- 0x556cfc86d350 con 0x556cfc84a150 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f1615ffb640 1 -- 172.21.15.107:0/1147506869 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f161c027030 con 0x556cfc84a150 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 10 client.25276 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 10 client.25276 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 20 client.25276 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:46.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc86e6a0 0x556cfc870a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 4 client.25276 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cfc7c2830 con 0x556cfc86e6a0 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.600+0000 7f1632349640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc86e6a0 0x556cfc870a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.601+0000 7f1615ffb640 10 client.25276 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.601+0000 7f1632349640 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc86e6a0 0x556cfc870a90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a60f62b7dff92711 server_cookie=a09d0660c7a9eb55 in_seq=0 out_seq=0 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.601+0000 7f1615ffb640 1 -- 172.21.15.107:0/1147506869 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cfc7c2830 con 0x556cfc86e6a0 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.601+0000 7f1615ffb640 10 client.25276 handle_command_reply: tid=0 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.601+0000 7f16369a6740 1 client.25276 shutdown 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 msgr2=0x556cfc8535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f161c0044c0 tx=0x7f161c0044f0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 shutdown_connections 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc853b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc86e6a0 0x556cfc870a90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc8535e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 --2- 172.21.15.107:0/1147506869 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 >> 172.21.15.107:0/1147506869 conn(0x556cfc847660 msgr2=0x556cfc84d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f1615ffb640 0 client.25276 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 shutdown_connections 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 1 -- 172.21.15.107:0/1147506869 wait complete. 2024-08-20T06:39:46.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.602+0000 7f16369a6740 20 client.25276 trim_cache size 0 max 16384 2024-08-20T06:39:46.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 Processor -- start 2024-08-20T06:39:46.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 -- start start 2024-08-20T06:39:46.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc853780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc8560e0 0x556cfc8761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1610010cd0 con 0x556cfc8560e0 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f161000e5a0 con 0x556cfc849830 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f16369a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f161000c3d0 con 0x556cfc84a150 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f1632349640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.624+0000 7f1632349640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:35454/0 (socket says 172.21.15.107:35454) 2024-08-20T06:39:46.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632b4a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc8560e0 0x556cfc8761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1631b48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc853780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 -- 172.21.15.107:0/2539445663 learned_addr learned my addr 172.21.15.107:0/2539445663 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:46.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 -- 172.21.15.107:0/2539445663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 msgr2=0x556cfc853780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc853780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 -- 172.21.15.107:0/2539445663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc8560e0 msgr2=0x556cfc8761d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc8560e0 0x556cfc8761d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 -- 172.21.15.107:0/2539445663 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cfc849f80 con 0x556cfc849830 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1632349640 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1628011ca0 tx=0x7f1628019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cda0f9ecd093cfc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.625+0000 7f1617fff640 1 -- 172.21.15.107:0/2539445663 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162801ee60 con 0x556cfc849830 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f1617fff640 1 -- 172.21.15.107:0/2539445663 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16280035f0 con 0x556cfc849830 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f1617fff640 1 -- 172.21.15.107:0/2539445663 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f162802a660 con 0x556cfc849830 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 -- 172.21.15.107:0/2539445663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 msgr2=0x556cfc852930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1628011ca0 tx=0x7f1628019000 comp rx=0 tx=0).stop 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 -- 172.21.15.107:0/2539445663 shutdown_connections 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc8560e0 0x556cfc8761d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc853780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 --2- 172.21.15.107:0/2539445663 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc849830 0x556cfc852930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.626+0000 7f16369a6740 1 -- 172.21.15.107:0/2539445663 >> 172.21.15.107:0/2539445663 conn(0x556cfc847660 msgr2=0x556cfc853070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.627+0000 7f16369a6740 1 -- 172.21.15.107:0/2539445663 shutdown_connections 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.627+0000 7f16369a6740 1 -- 172.21.15.107:0/2539445663 wait complete. 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.627+0000 7f16369a6740 1 -- start start 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.627+0000 7f16369a6740 1 Processor -- start 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8620f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc862630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f161000ca60 con 0x556cfc849830 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f161000ded0 con 0x556cfc8560e0 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f161000d130 con 0x556cfc84a150 2024-08-20T06:39:46.636 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f16369a6740 10 client.0 _ll_register_callbacks cb 0x7f162800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1632349640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8620f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc862630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1632349640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8620f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58278/0 (socket says 172.21.15.107:58278) 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1632349640 1 -- 172.21.15.107:0/3885700419 learned_addr learned my addr 172.21.15.107:0/3885700419 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1632b4a640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 -- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 msgr2=0x556cfc8d7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8d7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 -- 172.21.15.107:0/3885700419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 msgr2=0x556cfc8620f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8620f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 -- 172.21.15.107:0/3885700419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cfc8d7880 con 0x556cfc84a150 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1631b48640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc862630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f161c00f850 tx=0x7f161c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3af986c2be9fc5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:39:46.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.628+0000 7f1615ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161c0044f0 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16280035f0 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f161c011020 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f161c011220 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 10 client.25282 fetch_fsmap learned FSMap version 27 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556cfc8d8b70 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f16280035f0 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f161c027040 con 0x556cfc84a150 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 10 client.25282 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 20 client.25282 populate_metadata read hostname 'smithi107' 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc8d92f0 0x556cfc8db710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:39:46.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 4 client.25282 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.629+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556cfc7be2d0 con 0x556cfc8d92f0 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f1632349640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc8d92f0 0x556cfc8db710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f1632349640 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc8d92f0 0x556cfc8db710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be52d65267a0fbc3 server_cookie=101d644eb2d37c5f in_seq=0 out_seq=0 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f1615ffb640 10 client.25282 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f1615ffb640 1 -- 172.21.15.107:0/3885700419 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556cfc7be2d0 con 0x556cfc8d92f0 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f1615ffb640 10 client.25282 handle_command_reply: tid=0 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.630+0000 7f16369a6740 1 client.25282 shutdown 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 msgr2=0x556cfc862630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc862630 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f161c00f850 tx=0x7f161c0046c0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 shutdown_connections 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556cfc8560e0 0x556cfc8d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556cfc8d92f0 0x556cfc8db710 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556cfc84a150 0x556cfc862630 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 --2- 172.21.15.107:0/3885700419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556cfc849830 0x556cfc8620f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 >> 172.21.15.107:0/3885700419 conn(0x556cfc847660 msgr2=0x556cfc853830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f1615ffb640 0 client.25282 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 shutdown_connections 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 1 -- 172.21.15.107:0/3885700419 wait complete. 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.631+0000 7f16369a6740 20 client.25282 trim_cache size 0 max 16384 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:39:46.640 INFO:teuthology.orchestra.run.smithi107.stdout: "d9d50c8e-8c2c-4a6b-bb1e-a883122b1f0e": { 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d9d50c8e-8c2c-4a6b-bb1e-a883122b1f0e", 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:39:46.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 msgr2=0x7f16180b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 0x7f16180b6110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x556cfc84c9b0 tx=0x7f162800b410 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 msgr2=0x7f162c1a0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1624031040 tx=0x7f1624003b80 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 shutdown_connections 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f162c10ce20 0x7f162c1a0910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f162c104270 0x7f162c19b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f16180b3c50 0x7f16180b6110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 --2- 172.21.15.107:0/3103460235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f162c103870 0x7f162c19b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 >> 172.21.15.107:0/3103460235 conn(0x7f162c0fd400 msgr2=0x7f162c10efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 shutdown_connections 2024-08-20T06:39:46.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:39:46.632+0000 7f16345ad640 1 -- 172.21.15.107:0/3103460235 wait complete. 2024-08-20T06:39:47.387 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:47 smithi183 ceph-mon[22204]: pgmap v415: 97 pgs: 97 active+clean; 141 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-08-20T06:39:47.387 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:47 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:47.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:47 smithi193 ceph-mon[21944]: pgmap v415: 97 pgs: 97 active+clean; 141 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-08-20T06:39:47.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:47 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:47.596 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:47 smithi107 ceph-mon[18495]: pgmap v415: 97 pgs: 97 active+clean; 141 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 486 KiB/s rd, 1.3 MiB/s wr, 107 op/s 2024-08-20T06:39:47.597 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:47 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:39:47.633 INFO:tasks.cephfs.filesystem:scrub status for tag:d9d50c8e-8c2c-4a6b-bb1e-a883122b1f0e - {'path': '/', 'tag': 'd9d50c8e-8c2c-4a6b-bb1e-a883122b1f0e', 'options': 'recursive,force'} 2024-08-20T06:39:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:49 smithi183 ceph-mon[22204]: pgmap v416: 97 pgs: 97 active+clean; 146 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-08-20T06:39:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:49 smithi193 ceph-mon[21944]: pgmap v416: 97 pgs: 97 active+clean; 146 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-08-20T06:39:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:49 smithi107 ceph-mon[18495]: pgmap v416: 97 pgs: 97 active+clean; 146 MiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.7 MiB/s wr, 127 op/s 2024-08-20T06:39:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:51 smithi183 ceph-mon[22204]: pgmap v417: 97 pgs: 97 active+clean; 147 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:39:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:51 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:39:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:51 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:39:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:51 smithi193 ceph-mon[21944]: pgmap v417: 97 pgs: 97 active+clean; 147 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:39:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:51 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:39:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:51 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:39:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:51 smithi107 ceph-mon[18495]: pgmap v417: 97 pgs: 97 active+clean; 147 MiB data, 889 MiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.4 MiB/s wr, 108 op/s 2024-08-20T06:39:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:51 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:39:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:51 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:39:52.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:52 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:39:52.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:52 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:39:52.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:52 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:39:53.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:53 smithi183 ceph-mon[22204]: pgmap v418: 97 pgs: 97 active+clean; 150 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-08-20T06:39:53.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:53 smithi107 ceph-mon[18495]: pgmap v418: 97 pgs: 97 active+clean; 150 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-08-20T06:39:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:53 smithi193 ceph-mon[21944]: pgmap v418: 97 pgs: 97 active+clean; 150 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-08-20T06:39:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:55 smithi183 ceph-mon[22204]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-08-20T06:39:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:55 smithi193 ceph-mon[21944]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-08-20T06:39:55.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:55 smithi107 ceph-mon[18495]: pgmap v419: 97 pgs: 97 active+clean; 152 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-08-20T06:39:55.885 INFO:tasks.workunit.client.0.smithi107.stdout: 37 16394 187 9853 98 2922 242 2024-08-20T06:39:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:57 smithi183 ceph-mon[22204]: pgmap v420: 97 pgs: 97 active+clean; 157 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-08-20T06:39:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:57 smithi193 ceph-mon[21944]: pgmap v420: 97 pgs: 97 active+clean; 157 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-08-20T06:39:57.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:57 smithi107 ceph-mon[18495]: pgmap v420: 97 pgs: 97 active+clean; 157 MiB data, 902 MiB used, 1.0 TiB / 1.0 TiB avail; 773 KiB/s rd, 1.6 MiB/s wr, 124 op/s 2024-08-20T06:39:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:39:59 smithi183 ceph-mon[22204]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-20T06:39:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:39:59 smithi193 ceph-mon[21944]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-20T06:39:59.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:39:59 smithi107 ceph-mon[18495]: pgmap v421: 97 pgs: 97 active+clean; 159 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 765 KiB/s rd, 1.6 MiB/s wr, 129 op/s 2024-08-20T06:40:00.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:00 smithi183 ceph-mon[22204]: overall HEALTH_OK 2024-08-20T06:40:00.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:00 smithi193 ceph-mon[21944]: overall HEALTH_OK 2024-08-20T06:40:00.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:00 smithi107 ceph-mon[18495]: overall HEALTH_OK 2024-08-20T06:40:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:01 smithi183 ceph-mon[22204]: pgmap v422: 97 pgs: 97 active+clean; 161 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-08-20T06:40:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:01 smithi193 ceph-mon[21944]: pgmap v422: 97 pgs: 97 active+clean; 161 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-08-20T06:40:01.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:01 smithi107 ceph-mon[18495]: pgmap v422: 97 pgs: 97 active+clean; 161 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-08-20T06:40:03.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:03 smithi183 ceph-mon[22204]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T06:40:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:03 smithi193 ceph-mon[21944]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T06:40:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:03 smithi107 ceph-mon[18495]: pgmap v423: 97 pgs: 97 active+clean; 167 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T06:40:05.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:05 smithi183 ceph-mon[22204]: pgmap v424: 97 pgs: 97 active+clean; 171 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-08-20T06:40:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:05 smithi193 ceph-mon[21944]: pgmap v424: 97 pgs: 97 active+clean; 171 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-08-20T06:40:05.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:05 smithi107 ceph-mon[18495]: pgmap v424: 97 pgs: 97 active+clean; 171 MiB data, 942 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.9 MiB/s wr, 161 op/s 2024-08-20T06:40:05.885 INFO:tasks.workunit.client.0.smithi107.stdout: 40 15733 226 9580 174 3017 341 2024-08-20T06:40:07.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:07 smithi183 ceph-mon[22204]: pgmap v425: 97 pgs: 97 active+clean; 174 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-08-20T06:40:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:07 smithi193 ceph-mon[21944]: pgmap v425: 97 pgs: 97 active+clean; 174 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-08-20T06:40:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:07 smithi107 ceph-mon[18495]: pgmap v425: 97 pgs: 97 active+clean; 174 MiB data, 935 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-08-20T06:40:09.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:09 smithi183 ceph-mon[22204]: pgmap v426: 97 pgs: 97 active+clean; 180 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 159 op/s 2024-08-20T06:40:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:09 smithi193 ceph-mon[21944]: pgmap v426: 97 pgs: 97 active+clean; 180 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 159 op/s 2024-08-20T06:40:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:09 smithi107 ceph-mon[18495]: pgmap v426: 97 pgs: 97 active+clean; 180 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 2.1 MiB/s wr, 159 op/s 2024-08-20T06:40:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:11 smithi183 ceph-mon[22204]: pgmap v427: 97 pgs: 97 active+clean; 182 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-08-20T06:40:11.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:11 smithi193 ceph-mon[21944]: pgmap v427: 97 pgs: 97 active+clean; 182 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-08-20T06:40:11.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:11 smithi107 ceph-mon[18495]: pgmap v427: 97 pgs: 97 active+clean; 182 MiB data, 972 MiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 157 op/s 2024-08-20T06:40:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:40:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:40:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:40:12.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:40:12.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:40:12.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:40:12.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:40:12.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:40:12.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:40:12.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:40:12.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:40:12.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:40:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:13 smithi183 ceph-mon[22204]: pgmap v428: 97 pgs: 97 active+clean; 187 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-08-20T06:40:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:13 smithi193 ceph-mon[21944]: pgmap v428: 97 pgs: 97 active+clean; 187 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-08-20T06:40:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:13 smithi107 ceph-mon[18495]: pgmap v428: 97 pgs: 97 active+clean; 187 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 165 op/s 2024-08-20T06:40:15.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:15 smithi183 ceph-mon[22204]: pgmap v429: 97 pgs: 97 active+clean; 194 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T06:40:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:15 smithi193 ceph-mon[21944]: pgmap v429: 97 pgs: 97 active+clean; 194 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T06:40:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:15 smithi107 ceph-mon[18495]: pgmap v429: 97 pgs: 97 active+clean; 194 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T06:40:15.885 INFO:tasks.workunit.client.0.smithi107.stdout: 44 15028 132 9180 201 3403 330 2024-08-20T06:40:17.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:17 smithi183 ceph-mon[22204]: pgmap v430: 97 pgs: 97 active+clean; 195 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-08-20T06:40:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:17 smithi193 ceph-mon[21944]: pgmap v430: 97 pgs: 97 active+clean; 195 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-08-20T06:40:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:17 smithi107 ceph-mon[18495]: pgmap v430: 97 pgs: 97 active+clean; 195 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 132 op/s 2024-08-20T06:40:17.634 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:40:17.939 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:18.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 -- 172.21.15.107:0/1500620898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010b050 msgr2=0x7ff8e0111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 --2- 172.21.15.107:0/1500620898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010b050 0x7ff8e0111850 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8009be0 tx=0x7ff8d802a6c0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 -- 172.21.15.107:0/1500620898 shutdown_connections 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 --2- 172.21.15.107:0/1500620898 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010b050 0x7ff8e0111850 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 --2- 172.21.15.107:0/1500620898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010a6d0 0x7ff8e010ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 --2- 172.21.15.107:0/1500620898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e01050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.472+0000 7ff8e5965640 1 -- 172.21.15.107:0/1500620898 >> 172.21.15.107:0/1500620898 conn(0x7ff8e01006f0 msgr2=0x7ff8e0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.473+0000 7ff8e5965640 1 -- 172.21.15.107:0/1500620898 shutdown_connections 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.473+0000 7ff8e5965640 1 -- 172.21.15.107:0/1500620898 wait complete. 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.473+0000 7ff8e5965640 1 Processor -- start 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 -- start start 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e019b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 0x7ff8e01a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114b80 con 0x7ff8e0104cc0 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114a00 con 0x7ff8e010a6d0 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8e5965640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e0114d00 con 0x7ff8e010b050 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8deffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e019b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8de7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8de7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60092/0 (socket says 172.21.15.107:60092) 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 0x7ff8e01a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.474+0000 7ff8de7fc640 1 -- 172.21.15.107:0/2220122600 learned_addr learned my addr 172.21.15.107:0/2220122600 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 -- 172.21.15.107:0/2220122600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 msgr2=0x7ff8e01a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.535 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 0x7ff8e01a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 -- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 msgr2=0x7ff8e019b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e019b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 -- 172.21.15.107:0/2220122600 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8e01a1160 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 0x7ff8e01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8deffd640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400e9f0 tx=0x7ff8d400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8e4963640 1 -- 172.21.15.107:0/2220122600 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d400cd80 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8e4963640 1 -- 172.21.15.107:0/2220122600 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8d40108c0 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8e4963640 1 -- 172.21.15.107:0/2220122600 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40187c0 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8e01a1450 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.475+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8e01a7b10 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.478+0000 7ff8e4963640 1 -- 172.21.15.107:0/2220122600 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff8d4010430 con 0x7ff8e010a6d0 2024-08-20T06:40:18.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 Processor -- start 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 -- start start 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 0x55e2ab089a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab08c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2aaff48e0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2aafe3de0 con 0x55e2ab089f90 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.479+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2aafe7e20 con 0x55e2ab07e000 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48472/0 (socket says 172.21.15.107:48472) 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 -- 172.21.15.107:0/3028641091 learned_addr learned my addr 172.21.15.107:0/3028641091 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab08c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.537 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8deffd640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 0x55e2ab089a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8e4963640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 0x7ff8c80b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 -- 172.21.15.107:0/3028641091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 msgr2=0x55e2ab089a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 0x55e2ab089a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 -- 172.21.15.107:0/3028641091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 msgr2=0x55e2ab08c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab08c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 -- 172.21.15.107:0/3028641091 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2aad43650 con 0x55e2ab07d6e0 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8deffd640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 0x55e2ab089a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab08c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8e4963640 1 -- 172.21.15.107:0/2220122600 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff8d40e3eb0 con 0x7ff8e010a6d0 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8033000 tx=0x7ff8d8005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c952b4cf880697 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.480+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 0x7ff8c80b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.538 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.481+0000 7ff8c5ffb640 1 -- 172.21.15.107:0/3028641091 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8041070 con 0x55e2ab07d6e0 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.481+0000 7ff8c5ffb640 1 -- 172.21.15.107:0/3028641091 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8d8037040 con 0x55e2ab07d6e0 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8c5ffb640 1 -- 172.21.15.107:0/3028641091 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8036040 con 0x55e2ab07d6e0 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3028641091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 msgr2=0x55e2ab07dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 0x7ff8c80b6110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80049d0 tx=0x7ff8d8046550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8033000 tx=0x7ff8d8005eb0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3028641091 shutdown_connections 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab08c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07e000 0x55e2ab089a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3028641091 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07dac0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3028641091 >> 172.21.15.107:0/3028641091 conn(0x55e2ab07b510 msgr2=0x55e2ab07c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3028641091 shutdown_connections 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3028641091 wait complete. 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.482+0000 7ff8e7d5e740 1 -- start start 2024-08-20T06:40:18.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 Processor -- start 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab086ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab087670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8deffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8deffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60100/0 (socket says 172.21.15.107:60100) 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8deffd640 1 -- 172.21.15.107:0/3961119522 learned_addr learned my addr 172.21.15.107:0/3961119522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab086ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2aae46700 con 0x55e2ab07e000 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2aaf6a7c0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.483+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2ab08cc30 con 0x55e2ab089f90 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8e7d5e740 10 client.0 _ll_register_callbacks cb 0x55e2aafdd6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:18.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab087670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 -- 172.21.15.107:0/3961119522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 msgr2=0x55e2ab087670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab087670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 -- 172.21.15.107:0/3961119522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 msgr2=0x55e2ab086ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab086ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ab095fe0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab086ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8deffd640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0012d50 tx=0x7ff8d000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d6775779104ccea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8beffd640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00120d0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff8d0010430 con 0x55e2ab07d6e0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.484+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2aaf93100 con 0x55e2ab07d6e0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8d0012270 con 0x55e2ab07d6e0 2024-08-20T06:40:18.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 10 client.25388 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e2ab0989d0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e2aaf93100 con 0x55e2ab07d6e0 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8d001c4b0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 10 client.25388 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 20 client.25388 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab09c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 4 client.25388 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2aaff6830 con 0x55e2ab099e80 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.485+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab09c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.486+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab09c340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc24f35bcb70e08 server_cookie=88f545f619b57323 in_seq=0 out_seq=0 2024-08-20T06:40:18.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.486+0000 7ff8beffd640 10 client.25388 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.487+0000 7ff8beffd640 1 -- 172.21.15.107:0/3961119522 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2aaff6830 con 0x55e2ab099e80 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.487+0000 7ff8beffd640 10 client.25388 handle_command_reply: tid=0 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.487+0000 7ff8e7d5e740 1 client.25388 shutdown 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.487+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 msgr2=0x55e2ab089a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.487+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0012d50 tx=0x7ff8d000f790 comp rx=0 tx=0).stop 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 shutdown_connections 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab09c340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8beffd640 0 client.25388 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab087670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab086ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/3961119522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab089a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 >> 172.21.15.107:0/3961119522 conn(0x55e2ab07b510 msgr2=0x55e2ab0810b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 shutdown_connections 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/3961119522 wait complete. 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.488+0000 7ff8e7d5e740 20 client.25388 trim_cache size 0 max 16384 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 Processor -- start 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 -- start start 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab0817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab0a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01b2b30 con 0x55e2ab07e000 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01b0400 con 0x55e2ab089f90 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8e7d5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01ae230 con 0x55e2ab07d6e0 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8de7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab0817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8de7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab0817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48484/0 (socket says 172.21.15.107:48484) 2024-08-20T06:40:18.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8de7fc640 1 -- 172.21.15.107:0/1398923933 learned_addr learned my addr 172.21.15.107:0/1398923933 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab0a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.513+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 -- 172.21.15.107:0/1398923933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 msgr2=0x55e2ab0a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab0a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 -- 172.21.15.107:0/1398923933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 msgr2=0x55e2ab0817e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab0817e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 -- 172.21.15.107:0/1398923933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ab0879e0 con 0x55e2ab07d6e0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab0a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07aed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009530 tx=0x7ff8d0012990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ba073467760c0a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8c57fa640 1 -- 172.21.15.107:0/1398923933 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d001cd20 con 0x55e2ab07d6e0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8c57fa640 1 -- 172.21.15.107:0/1398923933 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff8d002b020 con 0x55e2ab07d6e0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.514+0000 7ff8c57fa640 1 -- 172.21.15.107:0/1398923933 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0005490 con 0x55e2ab07d6e0 2024-08-20T06:40:18.545 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.515+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1398923933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 msgr2=0x55e2ab07aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.515+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07aed0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009530 tx=0x7ff8d0012990 comp rx=0 tx=0).stop 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1398923933 shutdown_connections 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab089f90 0x55e2ab0a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab0817e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1398923933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab07d6e0 0x55e2ab07aed0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1398923933 >> 172.21.15.107:0/1398923933 conn(0x55e2ab07b510 msgr2=0x55e2ab085720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1398923933 shutdown_connections 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1398923933 wait complete. 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 -- start start 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.516+0000 7ff8e7d5e740 1 Processor -- start 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.546 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab092480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8deffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8deffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60112/0 (socket says 172.21.15.107:60112) 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8deffd640 1 -- 172.21.15.107:0/1309100519 learned_addr learned my addr 172.21.15.107:0/1309100519 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab092480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab083210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01ae8c0 con 0x55e2ab07e000 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01afd30 con 0x55e2ab07d6e0 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff8e01aef90 con 0x55e2ab089f90 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.517+0000 7ff8df7fe640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab083210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8e7d5e740 10 client.0 _ll_register_callbacks cb 0x7ff8d0009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 -- 172.21.15.107:0/1309100519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 msgr2=0x55e2ab083210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab083210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 -- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 msgr2=0x55e2ab091f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ab092b90 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8d002b020 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8de7fc640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab092480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40a7450 tx=0x7ff8d40a6e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8a4c1a0eeee22 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8beffd640 10 client.15876 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8beffd640 1 -- 172.21.15.107:0/1309100519 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4010720 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8beffd640 1 -- 172.21.15.107:0/1309100519 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff8d40a74a0 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8beffd640 1 -- 172.21.15.107:0/1309100519 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff8d40aa9c0 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8e7d5e740 10 client.15876 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.518+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=27}) -- 0x55e2ab10c910 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8beffd640 1 -- 172.21.15.107:0/1309100519 <== mon.0 v2:172.21.15.107:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff8d40aa110 con 0x55e2ab07e000 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:18.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 20 client.15876 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 4 client.15876 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e2aaff22d0 con 0x55e2ab099e80 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.519+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8beffd640 10 client.15876 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8deffd640 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab10efd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1d5a9301d592b11 server_cookie=dfc2ee8e6e6f56b4 in_seq=0 out_seq=0 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8beffd640 1 -- 172.21.15.107:0/1309100519 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e2aaff22d0 con 0x55e2ab099e80 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8beffd640 10 client.15876 handle_command_reply: tid=0 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 client.15876 shutdown 2024-08-20T06:40:18.549 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 msgr2=0x55e2ab092480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab092480 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40a7450 tx=0x7ff8d40a6e90 comp rx=0 tx=0).stop 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 shutdown_connections 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e2ab099e80 0x55e2ab10efd0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2ab089f90 0x55e2ab083210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.550 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2ab07e000 0x55e2ab092480 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 --2- 172.21.15.107:0/1309100519 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2ab07d6e0 0x55e2ab091f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 >> 172.21.15.107:0/1309100519 conn(0x55e2ab07b510 msgr2=0x55e2ab0884d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8beffd640 0 client.15876 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 shutdown_connections 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 1 -- 172.21.15.107:0/1309100519 wait complete. 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.520+0000 7ff8e7d5e740 20 client.15876 trim_cache size 0 max 16384 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 msgr2=0x7ff8c80b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 0x7ff8c80b6110 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80049d0 tx=0x7ff8d8046550 comp rx=0 tx=0).stop 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 msgr2=0x7ff8e019b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400e9f0 tx=0x7ff8d400eeb0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 shutdown_connections 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff8c80b3c50 0x7ff8c80b6110 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8e010b050 0x7ff8e01a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff8e010a6d0 0x7ff8e019b5f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 --2- 172.21.15.107:0/2220122600 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff8e0104cc0 0x7ff8e019b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 >> 172.21.15.107:0/2220122600 conn(0x7ff8e01006f0 msgr2=0x7ff8e0101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 shutdown_connections 2024-08-20T06:40:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:18.521+0000 7ff8e5965640 1 -- 172.21.15.107:0/2220122600 wait complete. 2024-08-20T06:40:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:19 smithi107 ceph-mon[18495]: pgmap v431: 97 pgs: 97 active+clean; 201 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-08-20T06:40:19.471 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:40:19.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:40:19.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:19 smithi183 ceph-mon[22204]: pgmap v431: 97 pgs: 97 active+clean; 201 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-08-20T06:40:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:19 smithi193 ceph-mon[21944]: pgmap v431: 97 pgs: 97 active+clean; 201 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 135 op/s 2024-08-20T06:40:19.876 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:20.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 -- 172.21.15.107:0/1724756579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a81073a0 msgr2=0x7f29a81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:20.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 --2- 172.21.15.107:0/1724756579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a81073a0 0x7f29a81077e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f299c009900 tx=0x7f299c02a410 comp rx=0 tx=0).stop 2024-08-20T06:40:20.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 -- 172.21.15.107:0/1724756579 shutdown_connections 2024-08-20T06:40:20.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 --2- 172.21.15.107:0/1724756579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8107d20 0x7f29a810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 --2- 172.21.15.107:0/1724756579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a81073a0 0x7f29a81077e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 --2- 172.21.15.107:0/1724756579 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a8101990 0x7f29a8101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 -- 172.21.15.107:0/1724756579 >> 172.21.15.107:0/1724756579 conn(0x7f29a80fd400 msgr2=0x7f29a80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 -- 172.21.15.107:0/1724756579 shutdown_connections 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.452+0000 7f29ac991640 1 -- 172.21.15.107:0/1724756579 wait complete. 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 Processor -- start 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 -- start start 2024-08-20T06:40:20.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 0x7f29a8198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 0x7f29a8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f29a8111820 con 0x7f29a8107d20 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81116a0 con 0x7f29a81073a0 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.453+0000 7f29ac991640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f29a81119a0 con 0x7f29a8101990 2024-08-20T06:40:20.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 0x7f29a8198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48506/0 (socket says 172.21.15.107:48506) 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 -- 172.21.15.107:0/2827124249 learned_addr learned my addr 172.21.15.107:0/2827124249 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 -- 172.21.15.107:0/2827124249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 msgr2=0x7f29a8198f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a5d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 0x7f29a8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:20.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 0x7f29a8198f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 -- 172.21.15.107:0/2827124249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 msgr2=0x7f29a8199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:20.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 0x7f29a8199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a6d9d640 1 -- 172.21.15.107:0/2827124249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a819efb0 con 0x7f29a8107d20 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a659c640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 0x7f29a8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.454+0000 7f29a5d9b640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 0x7f29a8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29a6d9d640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29a000eb10 tx=0x7f29a000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0004b20 con 0x7f29a8107d20 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f29a0004cc0 con 0x7f29a8107d20 2024-08-20T06:40:20.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a0010690 con 0x7f29a8107d20 2024-08-20T06:40:20.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a819f2a0 con 0x7f29a8107d20 2024-08-20T06:40:20.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.455+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a81a5960 con 0x7f29a8107d20 2024-08-20T06:40:20.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.457+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a8101d70 con 0x7f29a8107d20 2024-08-20T06:40:20.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.459+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f29a0005000 con 0x7f29a8107d20 2024-08-20T06:40:20.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.460+0000 7f29977fe640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 0x7f29900b61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:20.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.460+0000 7f29a659c640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 0x7f29900b61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:20.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.460+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f29a00e4420 con 0x7f29a8107d20 2024-08-20T06:40:20.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.461+0000 7f29a659c640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 0x7f29900b61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2998009b50 tx=0x7f2998005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:20.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.466+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f29a0055e20 con 0x7f29a8107d20 2024-08-20T06:40:20.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.823+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29a819a030 con 0x7f29a8107d20 2024-08-20T06:40:20.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.824+0000 7f29977fe640 1 -- 172.21.15.107:0/2827124249 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7f29a00abaf0 con 0x7f29a8107d20 2024-08-20T06:40:20.825 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:40:20.825 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-08-20T06:39:16:059531+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:39:15.312343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:40:20.826 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-08-20T06:40:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 msgr2=0x7f29900b61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 0x7f29900b61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2998009b50 tx=0x7f2998005f90 comp rx=0 tx=0).stop 2024-08-20T06:40:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 msgr2=0x7f29a819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f29a000eb10 tx=0x7f29a000c2d0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 shutdown_connections 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f29a8107d20 0x7f29a819e7e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f29a81073a0 0x7f29a8199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f29900b3d20 0x7f29900b61e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 --2- 172.21.15.107:0/2827124249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29a8101990 0x7f29a8198f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 >> 172.21.15.107:0/2827124249 conn(0x7f29a80fd400 msgr2=0x7f29a8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 shutdown_connections 2024-08-20T06:40:20.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:20.827+0000 7f29ac991640 1 -- 172.21.15.107:0/2827124249 wait complete. 2024-08-20T06:40:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:21 smithi183 ceph-mon[22204]: pgmap v432: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-08-20T06:40:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:21 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2827124249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:40:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:21 smithi193 ceph-mon[21944]: pgmap v432: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-08-20T06:40:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:21 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2827124249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:40:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:21 smithi107 ceph-mon[18495]: pgmap v432: 97 pgs: 97 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 930 KiB/s rd, 2.1 MiB/s wr, 121 op/s 2024-08-20T06:40:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:21 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2827124249' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:40:21.677 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:40:22.039 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:22.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 -- 172.21.15.107:0/3188854852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4108e20 msgr2=0x7f24d410f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 --2- 172.21.15.107:0/3188854852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4108e20 0x7f24d410f620 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009ca0 tx=0x7f24cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 -- 172.21.15.107:0/3188854852 shutdown_connections 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 --2- 172.21.15.107:0/3188854852 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4108e20 0x7f24d410f620 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 --2- 172.21.15.107:0/3188854852 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 0x7f24d41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 --2- 172.21.15.107:0/3188854852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4102a90 0x7f24d4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 -- 172.21.15.107:0/3188854852 >> 172.21.15.107:0/3188854852 conn(0x7f24d40fe540 msgr2=0x7f24d4100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 -- 172.21.15.107:0/3188854852 shutdown_connections 2024-08-20T06:40:22.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.605+0000 7f24da89c640 1 -- 172.21.15.107:0/3188854852 wait complete. 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 Processor -- start 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 -- start start 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 0x7f24d4198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 0x7f24d4199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 0x7f24d419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41129a0 con 0x7f24d4102a90 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4112820 con 0x7f24d41084a0 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.606+0000 7f24da89c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d4112b20 con 0x7f24d4108e20 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 0x7f24d419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 0x7f24d4198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 0x7f24d419e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:59640/0 (socket says 172.21.15.107:59640) 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d8e39640 1 -- 172.21.15.107:0/1790598549 learned_addr learned my addr 172.21.15.107:0/1790598549 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 0x7f24d4199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 -- 172.21.15.107:0/1790598549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 msgr2=0x7f24d419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 0x7f24d419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 -- 172.21.15.107:0/1790598549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 msgr2=0x7f24d4199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 0x7f24d4199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 -- 172.21.15.107:0/1790598549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d419eee0 con 0x7f24d4102a90 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d3fff640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 0x7f24d4198e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e9f0 tx=0x7f24c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24d1ffb640 1 -- 172.21.15.107:0/1790598549 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c400cd80 con 0x7f24d4102a90 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.607+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d419f1d0 con 0x7f24d4102a90 2024-08-20T06:40:22.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.608+0000 7f24d1ffb640 1 -- 172.21.15.107:0/1790598549 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24c40035f0 con 0x7f24d4102a90 2024-08-20T06:40:22.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.608+0000 7f24d1ffb640 1 -- 172.21.15.107:0/1790598549 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c4010670 con 0x7f24d4102a90 2024-08-20T06:40:22.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.608+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d41a5a60 con 0x7f24d4102a90 2024-08-20T06:40:22.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24d1ffb640 1 -- 172.21.15.107:0/1790598549 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f24c4004000 con 0x7f24d4102a90 2024-08-20T06:40:22.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 Processor -- start 2024-08-20T06:40:22.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 -- start start 2024-08-20T06:40:22.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae19bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae1a130 0x561c2ae1c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c2ad84930 con 0x561c2ae0e1a0 2024-08-20T06:40:22.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c2ad73e30 con 0x561c2ae0d880 2024-08-20T06:40:22.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24dcc95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c2ad77e70 con 0x561c2ae1a130 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.611+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60146/0 (socket says 172.21.15.107:60146) 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 -- 172.21.15.107:0/2342899531 learned_addr learned my addr 172.21.15.107:0/2342899531 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d3fff640 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae19bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae1a130 0x561c2ae1c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 -- 172.21.15.107:0/2342899531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae1a130 msgr2=0x561c2ae1c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae1a130 0x561c2ae1c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 -- 172.21.15.107:0/2342899531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 msgr2=0x561c2ae19bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae19bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 -- 172.21.15.107:0/2342899531 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c2aad3650 con 0x561c2ae0d880 2024-08-20T06:40:22.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.612+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009dc0 tx=0x7f24cc005f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e8b8bd4dce62435 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24ca7fc640 1 -- 172.21.15.107:0/2342899531 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc031980 con 0x561c2ae0d880 2024-08-20T06:40:22.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24ca7fc640 1 -- 172.21.15.107:0/2342899531 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24cc036030 con 0x561c2ae0d880 2024-08-20T06:40:22.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24ca7fc640 1 -- 172.21.15.107:0/2342899531 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004310 con 0x561c2ae0d880 2024-08-20T06:40:22.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24d1ffb640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 0x7f24c00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24d3fff640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 0x7f24c00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.613+0000 7f24d1ffb640 1 -- 172.21.15.107:0/1790598549 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f24c40e3e70 con 0x7f24d4102a90 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24d3fff640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 0x7f24c00b6110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24c40a8030 tx=0x7f24c4078040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 -- 172.21.15.107:0/2342899531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 msgr2=0x561c2ae0dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009dc0 tx=0x7f24cc005f70 comp rx=0 tx=0).stop 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 -- 172.21.15.107:0/2342899531 shutdown_connections 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae1a130 0x561c2ae1c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae19bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2342899531 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae0d880 0x561c2ae0dc60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 -- 172.21.15.107:0/2342899531 >> 172.21.15.107:0/2342899531 conn(0x561c2ae0b6b0 msgr2=0x561c2ae0c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 -- 172.21.15.107:0/2342899531 shutdown_connections 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.614+0000 7f24dcc95740 1 -- 172.21.15.107:0/2342899531 wait complete. 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 -- start start 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 Processor -- start 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae17b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24dcc95740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561c2abd6700 con 0x561c2ae0e1a0 2024-08-20T06:40:22.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.615+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:59656/0 (socket says 172.21.15.107:59656) 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 -- 172.21.15.107:0/1460694112 learned_addr learned my addr 172.21.15.107:0/1460694112 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561c2acfa810 con 0x561c2ae1a130 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561c2ae21010 con 0x561c2ae0d880 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae17b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24dcc95740 10 client.0 _ll_register_callbacks cb 0x561c2ad6d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 -- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 msgr2=0x561c2ae17b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae17b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 -- 172.21.15.107:0/1460694112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 msgr2=0x561c2ae17630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae17630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c2ae2a420 con 0x561c2ae0d880 2024-08-20T06:40:22.622 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24d8e39640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02b0b0 tx=0x7f24cc02b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad87192c4b1da57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004d40 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f24cc004440 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.616+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c2ad23150 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24cc004640 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 10 client.25315 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c2ae2cd90 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561c2ad23150 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f24cc043070 con 0x561c2ae0d880 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 10 client.25315 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 10 client.25315 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 20 client.25315 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:22.623 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae2e0e0 0x561c2ae304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 4 client.25315 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.617+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c2ad86880 con 0x561c2ae2e0e0 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.618+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae2e0e0 0x561c2ae304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.618+0000 7f24b37fe640 10 client.25315 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.618+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae2e0e0 0x561c2ae304d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c8061de47313ef37 server_cookie=2453897cbf811c0c in_seq=0 out_seq=0 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24b37fe640 1 -- 172.21.15.107:0/1460694112 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c2ad86880 con 0x561c2ae2e0e0 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24b37fe640 10 client.25315 handle_command_reply: tid=0 2024-08-20T06:40:22.624 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 client.25315 shutdown 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 msgr2=0x561c2ae170f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f24cc02b0b0 tx=0x7f24cc02b040 comp rx=0 tx=0).stop 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 shutdown_connections 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae2e0e0 0x561c2ae304d0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae17b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae17630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 --2- 172.21.15.107:0/1460694112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae170f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.619+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 >> 172.21.15.107:0/1460694112 conn(0x561c2ae0b6b0 msgr2=0x561c2ae11250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.620+0000 7f24b37fe640 0 client.25315 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:40:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.620+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 shutdown_connections 2024-08-20T06:40:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.620+0000 7f24dcc95740 1 -- 172.21.15.107:0/1460694112 wait complete. 2024-08-20T06:40:22.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.620+0000 7f24dcc95740 20 client.25315 trim_cache size 0 max 16384 2024-08-20T06:40:22.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 Processor -- start 2024-08-20T06:40:22.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 -- start start 2024-08-20T06:40:22.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae35bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae36100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ae6a0 con 0x561c2ae0e1a0 2024-08-20T06:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41abf70 con 0x561c2ae1a130 2024-08-20T06:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24dcc95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41a9dc0 con 0x561c2ae0d880 2024-08-20T06:40:22.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24cb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae35bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:59662/0 (socket says 172.21.15.107:59662) 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24cb7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae35bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48532/0 (socket says 172.21.15.107:48532) 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24d8e39640 1 -- 172.21.15.107:0/3323603410 learned_addr learned my addr 172.21.15.107:0/3323603410 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.640+0000 7f24d8e39640 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae36100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 -- 172.21.15.107:0/3323603410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 msgr2=0x561c2ae36100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae36100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 -- 172.21.15.107:0/3323603410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 msgr2=0x561c2ae35bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae35bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 -- 172.21.15.107:0/3323603410 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c2ae0c130 con 0x561c2ae0d880 2024-08-20T06:40:22.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24d8e39640 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24cc031a90 tx=0x7f24cc000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6046ae4fffa2fe20 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24c9ffb640 1 -- 172.21.15.107:0/3323603410 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc009800 con 0x561c2ae0d880 2024-08-20T06:40:22.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24c9ffb640 1 -- 172.21.15.107:0/3323603410 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24cc036070 con 0x561c2ae0d880 2024-08-20T06:40:22.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.641+0000 7f24c9ffb640 1 -- 172.21.15.107:0/3323603410 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc054c90 con 0x561c2ae0d880 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- 172.21.15.107:0/3323603410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 msgr2=0x561c2ae16980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24cc031a90 tx=0x7f24cc000f40 comp rx=0 tx=0).stop 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- 172.21.15.107:0/3323603410 shutdown_connections 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae36100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae35bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 --2- 172.21.15.107:0/3323603410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae16980 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- 172.21.15.107:0/3323603410 >> 172.21.15.107:0/3323603410 conn(0x561c2ae0b6b0 msgr2=0x561c2ae18f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- 172.21.15.107:0/3323603410 shutdown_connections 2024-08-20T06:40:22.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- 172.21.15.107:0/3323603410 wait complete. 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.642+0000 7f24dcc95740 1 -- start start 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 Processor -- start 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae217d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41aa430 con 0x561c2ae0e1a0 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41ab8a0 con 0x561c2ae1a130 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f24d41aab00 con 0x561c2ae0d880 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24dcc95740 10 client.0 _ll_register_callbacks cb 0x7f24cc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24d8e39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:59678/0 (socket says 172.21.15.107:59678) 2024-08-20T06:40:22.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24d8e39640 1 -- 172.21.15.107:0/2517546336 learned_addr learned my addr 172.21.15.107:0/2517546336 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae217d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.643+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 -- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 msgr2=0x561c2ae217d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae217d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 -- 172.21.15.107:0/2517546336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 msgr2=0x561c2ae21290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae21290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 -- 172.21.15.107:0/2517546336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c2ae22000 con 0x561c2ae0d880 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24d8e39640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24cc031920 tx=0x7f24cc005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd36abebebcae99e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24cc009800 con 0x561c2ae0d880 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f24cc004440 con 0x561c2ae0d880 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24cc036070 con 0x561c2ae0d880 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f24cc031450 con 0x561c2ae0d880 2024-08-20T06:40:22.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24dcc95740 10 client.25321 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x561c2ae97be0 con 0x561c2ae0d880 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.644+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f24cc04b040 con 0x561c2ae0d880 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f24cc06c070 con 0x561c2ae0d880 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 10 client.25321 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 10 client.25321 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 20 client.25321 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae98270 0x561c2ae9a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 4 client.25321 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f24cc036070 con 0x561c2ae98270 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.645+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae98270 0x561c2ae9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24b37fe640 10 client.25321 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:40:22.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24cb7fe640 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae98270 0x561c2ae9a690 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d69c54278843f68b server_cookie=df5924f52db1c50e in_seq=0 out_seq=0 2024-08-20T06:40:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24b37fe640 1 -- 172.21.15.107:0/2517546336 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f24cc036070 con 0x561c2ae98270 2024-08-20T06:40:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24b37fe640 10 client.25321 handle_command_reply: tid=0 2024-08-20T06:40:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 client.25321 shutdown 2024-08-20T06:40:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 msgr2=0x561c2ae269a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f24cc031920 tx=0x7f24cc005f70 comp rx=0 tx=0).stop 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 shutdown_connections 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x561c2ae98270 0x561c2ae9a690 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561c2ae1a130 0x561c2ae217d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561c2ae0e1a0 0x561c2ae21290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 --2- 172.21.15.107:0/2517546336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561c2ae0d880 0x561c2ae269a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 >> 172.21.15.107:0/2517546336 conn(0x561c2ae0b6b0 msgr2=0x561c2ae10a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24b37fe640 0 client.25321 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 shutdown_connections 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 1 -- 172.21.15.107:0/2517546336 wait complete. 2024-08-20T06:40:22.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.646+0000 7f24dcc95740 20 client.25321 trim_cache size 0 max 16384 2024-08-20T06:40:22.653 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:40:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 msgr2=0x7f24c00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 0x7f24c00b6110 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f24c40a8030 tx=0x7f24c4078040 comp rx=0 tx=0).stop 2024-08-20T06:40:22.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 msgr2=0x7f24d4198e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 0x7f24d4198e30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f24c400e9f0 tx=0x7f24c400eeb0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 shutdown_connections 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f24c00b3c50 0x7f24c00b6110 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24d4108e20 0x7f24d419e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f24d41084a0 0x7f24d4199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 --2- 172.21.15.107:0/1790598549 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f24d4102a90 0x7f24d4198e30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.647+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 >> 172.21.15.107:0/1790598549 conn(0x7f24d40fe540 msgr2=0x7f24d40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.648+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 shutdown_connections 2024-08-20T06:40:22.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:22.648+0000 7f24da89c640 1 -- 172.21.15.107:0/1790598549 wait complete. 2024-08-20T06:40:23.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:23 smithi107 ceph-mon[18495]: pgmap v433: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 136 op/s 2024-08-20T06:40:23.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:23 smithi183 ceph-mon[22204]: pgmap v433: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 136 op/s 2024-08-20T06:40:23.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:40:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:23 smithi193 ceph-mon[21944]: pgmap v433: 97 pgs: 97 active+clean; 212 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 136 op/s 2024-08-20T06:40:23.836 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 -- 172.21.15.107:0/3945167301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 msgr2=0x7fdd701088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 --2- 172.21.15.107:0/3945167301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd701088e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009900 tx=0x7fdd6002a410 comp rx=0 tx=0).stop 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 -- 172.21.15.107:0/3945167301 shutdown_connections 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 --2- 172.21.15.107:0/3945167301 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd7010f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 --2- 172.21.15.107:0/3945167301 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd701088e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 --2- 172.21.15.107:0/3945167301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 0x7fdd70102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 -- 172.21.15.107:0/3945167301 >> 172.21.15.107:0/3945167301 conn(0x7fdd700fe540 msgr2=0x7fdd70100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 -- 172.21.15.107:0/3945167301 shutdown_connections 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.364+0000 7fdd787c2640 1 -- 172.21.15.107:0/3945167301 wait complete. 2024-08-20T06:40:24.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 Processor -- start 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 -- start start 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 0x7fdd7019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd7019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701129e0 con 0x7fdd701084a0 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd70112860 con 0x7fdd70108e20 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd787c2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd70112b60 con 0x7fdd70102a90 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd76d5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd7019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd76d5f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60196/0 (socket says 172.21.15.107:60196) 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd76d5f640 1 -- 172.21.15.107:0/2906024713 learned_addr learned my addr 172.21.15.107:0/2906024713 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:24.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.365+0000 7fdd7655e640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 0x7fdd7019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 -- 172.21.15.107:0/2906024713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 msgr2=0x7fdd7019b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 0x7fdd7019b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 -- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 msgr2=0x7fdd7019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd7019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 -- 172.21.15.107:0/2906024713 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd701a1200 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd76d5f640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800efd0 tx=0x7fdd6800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd677fe640 1 -- 172.21.15.107:0/2906024713 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68003b70 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd677fe640 1 -- 172.21.15.107:0/2906024713 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd68003d10 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd677fe640 1 -- 172.21.15.107:0/2906024713 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd68010850 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd701a14f0 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.366+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd701a7bb0 con 0x7fdd70108e20 2024-08-20T06:40:24.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.369+0000 7fdd7abbb740 1 Processor -- start 2024-08-20T06:40:24.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.370+0000 7fdd677fe640 1 -- 172.21.15.107:0/2906024713 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fdd68004000 con 0x7fdd70108e20 2024-08-20T06:40:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 -- start start 2024-08-20T06:40:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a372b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee08999930 con 0x55ee08a23050 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee08988e30 con 0x55ee08a377f0 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee0898ce70 con 0x55ee08a22730 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd7655e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a372b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:59688/0 (socket says 172.21.15.107:59688) 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3645483151 learned_addr learned my addr 172.21.15.107:0/3645483151 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.371+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd677fe640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 0x7fdd5c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3645483151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 msgr2=0x55ee08a39be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a39be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3645483151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 msgr2=0x55ee08a372b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a372b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3645483151 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee086e8650 con 0x55ee08a22730 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd76d5f640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 0x7fdd5c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009900 tx=0x7fdd60000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b906e09d80f05102 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.372+0000 7fdd64ff9640 1 -- 172.21.15.107:0/3645483151 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60035070 con 0x55ee08a22730 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.373+0000 7fdd64ff9640 1 -- 172.21.15.107:0/3645483151 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd60003730 con 0x55ee08a22730 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.373+0000 7fdd64ff9640 1 -- 172.21.15.107:0/3645483151 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60004920 con 0x55ee08a22730 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.373+0000 7fdd677fe640 1 -- 172.21.15.107:0/2906024713 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd680e3fc0 con 0x7fdd70108e20 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3645483151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 msgr2=0x55ee08a22b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009900 tx=0x7fdd60000bc0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd76d5f640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 0x7fdd5c0b6090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd680a6fe0 tx=0x7fdd680a6f70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3645483151 shutdown_connections 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a39be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a372b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3645483151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a22b10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3645483151 >> 172.21.15.107:0/3645483151 conn(0x55ee08a20560 msgr2=0x55ee08a21700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3645483151 shutdown_connections 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3645483151 wait complete. 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 -- start start 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.374+0000 7fdd7abbb740 1 Processor -- start 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a35320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ee087eb700 con 0x55ee08a23050 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ee0890f810 con 0x55ee08a377f0 2024-08-20T06:40:24.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ee08a3a410 con 0x55ee08a22730 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48570/0 (socket says 172.21.15.107:48570) 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 -- 172.21.15.107:0/621579394 learned_addr learned my addr 172.21.15.107:0/621579394 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7abbb740 10 client.0 _ll_register_callbacks cb 0x55ee08982700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a35320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 -- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 msgr2=0x55ee08a348a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a348a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 -- 172.21.15.107:0/621579394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 msgr2=0x55ee08a35320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a35320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 -- 172.21.15.107:0/621579394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee08a438a0 con 0x55ee08a23050 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:24.383 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd7655e640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0126d0 tx=0x7fdd6c012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66320f8728a8033 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.375+0000 7fdd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c011bb0 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ee08938150 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd6c011d50 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd6c00f460 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd7abbb740 10 client.15912 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee08a462c0 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ee08938150 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.376+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd6c01c5e0 con 0x55ee08a23050 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 10 client.15912 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 10 client.15912 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 20 client.15912 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:24.384 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08a47b40 0x55ee08a49f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 4 client.15912 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:24.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee0899b880 con 0x55ee08a47b40 2024-08-20T06:40:24.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.377+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08a47b40 0x55ee08a49f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08a47b40 0x55ee08a49f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efcb8f9d7b58e422 server_cookie=e31d531928038b53 in_seq=0 out_seq=0 2024-08-20T06:40:24.385 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd59ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd59ffb640 1 -- 172.21.15.107:0/621579394 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee0899b880 con 0x55ee08a47b40 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd59ffb640 10 client.15912 handle_command_reply: tid=0 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd7abbb740 1 client.15912 shutdown 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 msgr2=0x55ee08a34de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.378+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c0126d0 tx=0x7fdd6c012b90 comp rx=0 tx=0).stop 2024-08-20T06:40:24.386 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 shutdown_connections 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08a47b40 0x55ee08a49f80 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a35320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a34de0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/621579394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a22730 0x55ee08a348a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 >> 172.21.15.107:0/621579394 conn(0x55ee08a20560 msgr2=0x55ee08a26100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd59ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 shutdown_connections 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 1 -- 172.21.15.107:0/621579394 wait complete. 2024-08-20T06:40:24.387 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.379+0000 7fdd7abbb740 20 client.15912 trim_cache size 0 max 16384 2024-08-20T06:40:24.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 Processor -- start 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 -- start start 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 0x55ee08a21590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 0x55ee08a32280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701ae680 con 0x55ee08a22730 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701abf50 con 0x55ee08a377f0 2024-08-20T06:40:24.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701a9d80 con 0x55ee08a23050 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 0x55ee08a21590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 0x55ee08a21590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48582/0 (socket says 172.21.15.107:48582) 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd75d5d640 1 -- 172.21.15.107:0/1391002881 learned_addr learned my addr 172.21.15.107:0/1391002881 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.396+0000 7fdd7655e640 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 0x55ee08a32280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 -- 172.21.15.107:0/1391002881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 msgr2=0x55ee08a32280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 0x55ee08a32280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 -- 172.21.15.107:0/1391002881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 msgr2=0x55ee08a21590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 0x55ee08a21590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 -- 172.21.15.107:0/1391002881 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee08a32da0 con 0x55ee08a377f0 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd7655e640 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 0x55ee08a32280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a34da0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd6002a9d0 tx=0x7fdd60005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6e5f2b669af4b3c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd5bfff640 1 -- 172.21.15.107:0/1391002881 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60035040 con 0x55ee08a377f0 2024-08-20T06:40:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd5bfff640 1 -- 172.21.15.107:0/1391002881 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd6005e030 con 0x55ee08a377f0 2024-08-20T06:40:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.397+0000 7fdd5bfff640 1 -- 172.21.15.107:0/1391002881 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60005410 con 0x55ee08a377f0 2024-08-20T06:40:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 -- 172.21.15.107:0/1391002881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 msgr2=0x55ee08a34da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.402 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a34da0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd6002a9d0 tx=0x7fdd60005fd0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 -- 172.21.15.107:0/1391002881 shutdown_connections 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a377f0 0x55ee08a34da0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a23050 0x55ee08a32280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/1391002881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a22730 0x55ee08a21590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 -- 172.21.15.107:0/1391002881 >> 172.21.15.107:0/1391002881 conn(0x55ee08a20560 msgr2=0x55ee08a33120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 -- 172.21.15.107:0/1391002881 shutdown_connections 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.398+0000 7fdd7abbb740 1 -- 172.21.15.107:0/1391002881 wait complete. 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 -- start start 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 Processor -- start 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a3f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.403 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a377f0 0x55ee08a3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701aa410 con 0x55ee08a23050 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701ab880 con 0x55ee08a22730 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd701aaae0 con 0x55ee08a377f0 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd7abbb740 10 client.0 _ll_register_callbacks cb 0x7fdd6c0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.399+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd7655e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a3f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:60248/0 (socket says 172.21.15.107:60248) 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3207803601 learned_addr learned my addr 172.21.15.107:0/3207803601 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a377f0 msgr2=0x55ee08a3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.404 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a377f0 0x55ee08a3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3207803601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 msgr2=0x55ee08a3f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a3f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 -- 172.21.15.107:0/3207803601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee08a401d0 con 0x55ee08a22730 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd75d5d640 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd600057c0 tx=0x7fdd60009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=549b1b0753ac040d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.400+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd60067060 con 0x55ee08a22730 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd60005390 con 0x55ee08a22730 2024-08-20T06:40:24.405 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd6005e030 con 0x55ee08a22730 2024-08-20T06:40:24.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd60035070 con 0x55ee08a22730 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 10 client.25433 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ee08ab1820 con 0x55ee08a22730 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd6005e030 con 0x55ee08a22730 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd6006c070 con 0x55ee08a22730 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 10 client.25433 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 10 client.25433 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:40:24.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.401+0000 7fdd7abbb740 20 client.25433 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08ab1a40 0x55ee08ab3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7abbb740 4 client.25433 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee08997320 con 0x55ee08ab1a40 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7655e640 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08ab1a40 0x55ee08ab3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd59ffb640 10 client.25433 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7655e640 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08ab1a40 0x55ee08ab3e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38c3d83420401a34 server_cookie=8620c9360c7320bc in_seq=0 out_seq=0 2024-08-20T06:40:24.412 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd59ffb640 1 -- 172.21.15.107:0/3207803601 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee08997320 con 0x55ee08ab1a40 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd59ffb640 10 client.25433 handle_command_reply: tid=0 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.402+0000 7fdd7abbb740 1 client.25433 shutdown 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 msgr2=0x55ee08a4fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdd600057c0 tx=0x7fdd60009290 comp rx=0 tx=0).stop 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 shutdown_connections 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55ee08ab1a40 0x55ee08ab3e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ee08a377f0 0x55ee08a3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd59ffb640 0 client.25433 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ee08a23050 0x55ee08a3f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 --2- 172.21.15.107:0/3207803601 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ee08a22730 0x55ee08a4fc90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 >> 172.21.15.107:0/3207803601 conn(0x55ee08a20560 msgr2=0x55ee08a1ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 shutdown_connections 2024-08-20T06:40:24.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 1 -- 172.21.15.107:0/3207803601 wait complete. 2024-08-20T06:40:24.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.403+0000 7fdd7abbb740 20 client.25433 trim_cache size 0 max 16384 2024-08-20T06:40:24.415 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 msgr2=0x7fdd5c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 0x7fdd5c0b6090 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fdd680a6fe0 tx=0x7fdd680a6f70 comp rx=0 tx=0).stop 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 msgr2=0x7fdd701a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800efd0 tx=0x7fdd6800c6a0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 shutdown_connections 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd70108e20 0x7fdd701a0a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd701084a0 0x7fdd7019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd5c0b3bd0 0x7fdd5c0b6090 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 --2- 172.21.15.107:0/2906024713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd70102a90 0x7fdd7019b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:24.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.404+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 >> 172.21.15.107:0/2906024713 conn(0x7fdd700fe540 msgr2=0x7fdd70104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:24.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.405+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 shutdown_connections 2024-08-20T06:40:24.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:24.405+0000 7fdd787c2640 1 -- 172.21.15.107:0/2906024713 wait complete. 2024-08-20T06:40:25.257 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:40:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:25 smithi183 ceph-mon[22204]: pgmap v434: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-08-20T06:40:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:25 smithi193 ceph-mon[21944]: pgmap v434: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-08-20T06:40:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:25 smithi107 ceph-mon[18495]: pgmap v434: 97 pgs: 97 active+clean; 215 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 146 op/s 2024-08-20T06:40:25.637 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:25.886 INFO:tasks.workunit.client.0.smithi107.stdout: 46 14113 222 8595 150 2873 388 2024-08-20T06:40:26.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 -- 172.21.15.107:0/190986553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa941020b0 msgr2=0x7ffa94106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/190986553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa941020b0 0x7ffa94106580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffa80009900 tx=0x7ffa8002a3f0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 -- 172.21.15.107:0/190986553 shutdown_connections 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/190986553 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94106bd0 0x7ffa94107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/190986553 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa941020b0 0x7ffa94106580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/190986553 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa94101790 0x7ffa94101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.140+0000 7ffa9ae35640 1 -- 172.21.15.107:0/190986553 >> 172.21.15.107:0/190986553 conn(0x7ffa940fd400 msgr2=0x7ffa940ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.141+0000 7ffa9ae35640 1 -- 172.21.15.107:0/190986553 shutdown_connections 2024-08-20T06:40:26.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.141+0000 7ffa9ae35640 1 -- 172.21.15.107:0/190986553 wait complete. 2024-08-20T06:40:26.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.141+0000 7ffa9ae35640 1 Processor -- start 2024-08-20T06:40:26.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 -- start start 2024-08-20T06:40:26.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 0x7ffa9419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa94106bd0 0x7ffa941a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94118f40 con 0x7ffa94106bd0 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94118dc0 con 0x7ffa94101790 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa9ae35640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941190c0 con 0x7ffa941020b0 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa993d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa94106bd0 0x7ffa941a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:40872/0 (socket says 172.21.15.107:40872) 2024-08-20T06:40:26.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 -- 172.21.15.107:0/244890223 learned_addr learned my addr 172.21.15.107:0/244890223 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 0x7ffa9419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 -- 172.21.15.107:0/244890223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 msgr2=0x7ffa9419f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 0x7ffa9419f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 -- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa94106bd0 msgr2=0x7ffa941a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa94106bd0 0x7ffa941a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa93fff640 1 -- 172.21.15.107:0/244890223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa941a5580 con 0x7ffa941020b0 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.142+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 0x7ffa9419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa93fff640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002a910 tx=0x7ffa80003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa91ffb640 1 -- 172.21.15.107:0/244890223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8003f070 con 0x7ffa941020b0 2024-08-20T06:40:26.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa91ffb640 1 -- 172.21.15.107:0/244890223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa8003baf0 con 0x7ffa941020b0 2024-08-20T06:40:26.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa91ffb640 1 -- 172.21.15.107:0/244890223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80035030 con 0x7ffa941020b0 2024-08-20T06:40:26.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa941a5810 con 0x7ffa941020b0 2024-08-20T06:40:26.149 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.143+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa941abf30 con 0x7ffa941020b0 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.146+0000 7ffa91ffb640 1 -- 172.21.15.107:0/244890223 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ffa80042070 con 0x7ffa941020b0 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 Processor -- start 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 -- start start 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa900130 0x5619fa902520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.151 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619fa86a930 con 0x5619fa8f41a0 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5619fa859e30 con 0x5619fa900130 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619fa85de70 con 0x5619fa8f3880 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.147+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45298/0 (socket says 172.21.15.107:45298) 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa98bd1640 1 -- 172.21.15.107:0/622761668 learned_addr learned my addr 172.21.15.107:0/622761668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa93fff640 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa900130 0x5619fa902520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa91ffb640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 0x7ffa840b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 -- 172.21.15.107:0/622761668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa900130 msgr2=0x5619fa902520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa900130 0x5619fa902520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 -- 172.21.15.107:0/622761668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 msgr2=0x5619fa8ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 -- 172.21.15.107:0/622761668 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fa5b9650 con 0x5619fa8f3880 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:26.152 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa91ffb640 1 -- 172.21.15.107:0/244890223 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ffa80104270 con 0x7ffa941020b0 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.148+0000 7ffa993d2640 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f3c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009f60 tx=0x7ffa8c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8bab4945063afa3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.149+0000 7ffa993d2640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 0x7ffa840b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.149+0000 7ffa7effd640 1 -- 172.21.15.107:0/622761668 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c016070 con 0x5619fa8f3880 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.149+0000 7ffa7effd640 1 -- 172.21.15.107:0/622761668 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa8c01bbd0 con 0x5619fa8f3880 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.149+0000 7ffa7effd640 1 -- 172.21.15.107:0/622761668 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c012e50 con 0x5619fa8f3880 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- 172.21.15.107:0/622761668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 msgr2=0x5619fa8f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa993d2640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 0x7ffa840b6090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c01a000 tx=0x7ffa8c0114c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f3c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009f60 tx=0x7ffa8c009f90 comp rx=0 tx=0).stop 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- 172.21.15.107:0/622761668 shutdown_connections 2024-08-20T06:40:26.153 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa900130 0x5619fa902520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa8f41a0 0x5619fa8ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/622761668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f3c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- 172.21.15.107:0/622761668 >> 172.21.15.107:0/622761668 conn(0x5619fa8f16b0 msgr2=0x5619fa8f2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- 172.21.15.107:0/622761668 shutdown_connections 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- 172.21.15.107:0/622761668 wait complete. 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 -- start start 2024-08-20T06:40:26.154 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.150+0000 7ffa9d22e740 1 Processor -- start 2024-08-20T06:40:26.155 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa8fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5619fa6bc700 con 0x5619fa900130 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5619fa7e0810 con 0x5619fa8f3880 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5619fa902de0 con 0x5619fa8f41a0 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa9d22e740 10 client.0 _ll_register_callbacks cb 0x5619fa853700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa993d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa993d2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45310/0 (socket says 172.21.15.107:45310) 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa993d2640 1 -- 172.21.15.107:0/95028295 learned_addr learned my addr 172.21.15.107:0/95028295 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.151+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa8fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 -- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 msgr2=0x5619fa8fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa8fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.156 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 -- 172.21.15.107:0/95028295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 msgr2=0x5619fa8fdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 -- 172.21.15.107:0/95028295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fa90c160 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa993d2640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa93fff640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5619fa8f3830 tx=0x7ffa80009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=862c1e095c219ec5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa73fff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80049050 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619fa809150 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ffa800c7450 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa800ca930 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa9d22e740 10 client.25348 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.152+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619fa90eb10 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5619fa809150 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa800d3de0 con 0x5619fa8f41a0 2024-08-20T06:40:26.157 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 10 client.25348 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 10 client.25348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 20 client.25348 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa90ffc0 0x5619fa9123b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 4 client.25348 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619fa86c880 con 0x5619fa90ffc0 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.153+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa90ffc0 0x5619fa9123b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.154+0000 7ffa73fff640 10 client.25348 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:26.158 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.154+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa90ffc0 0x5619fa9123b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c7c2f80e8505c5 server_cookie=690b1ece8dfc472f in_seq=0 out_seq=0 2024-08-20T06:40:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa73fff640 1 -- 172.21.15.107:0/95028295 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619fa86c880 con 0x5619fa90ffc0 2024-08-20T06:40:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa73fff640 10 client.25348 handle_command_reply: tid=0 2024-08-20T06:40:26.159 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 client.25348 shutdown 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 msgr2=0x5619fa8fd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5619fa8f3830 tx=0x7ffa80009260 comp rx=0 tx=0).stop 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 shutdown_connections 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa90ffc0 0x5619fa9123b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/95028295 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa8fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 >> 172.21.15.107:0/95028295 conn(0x5619fa8f16b0 msgr2=0x5619fa8f7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.156+0000 7ffa73fff640 0 client.25348 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.157+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 shutdown_connections 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.157+0000 7ffa9d22e740 1 -- 172.21.15.107:0/95028295 wait complete. 2024-08-20T06:40:26.160 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.157+0000 7ffa9d22e740 20 client.25348 trim_cache size 0 max 16384 2024-08-20T06:40:26.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.175+0000 7ffa9d22e740 1 Processor -- start 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.175+0000 7ffa9d22e740 1 -- start start 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.175+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.175+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.175+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b6f70 con 0x5619fa900130 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b4840 con 0x5619fa8f41a0 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b2670 con 0x5619fa8f3880 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:45646/0 (socket says 172.21.15.107:45646) 2024-08-20T06:40:26.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:40898/0 (socket says 172.21.15.107:40898) 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 -- 172.21.15.107:0/1598581095 learned_addr learned my addr 172.21.15.107:0/1598581095 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa993d2640 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 -- 172.21.15.107:0/1598581095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 msgr2=0x5619fa8f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 -- 172.21.15.107:0/1598581095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 msgr2=0x5619fa8fdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 -- 172.21.15.107:0/1598581095 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619fa8f2130 con 0x5619fa8f41a0 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa93fff640 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002a9b0 tx=0x7ffa801044f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b9f4bc5bfd89e0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.176+0000 7ffa7e7fc640 1 -- 172.21.15.107:0/1598581095 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80049070 con 0x5619fa8f41a0 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa7e7fc640 1 -- 172.21.15.107:0/1598581095 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ffa80031400 con 0x5619fa8f41a0 2024-08-20T06:40:26.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa7e7fc640 1 -- 172.21.15.107:0/1598581095 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80107920 con 0x5619fa8f41a0 2024-08-20T06:40:26.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 -- 172.21.15.107:0/1598581095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 msgr2=0x5619fa8fd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002a9b0 tx=0x7ffa801044f0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 -- 172.21.15.107:0/1598581095 shutdown_connections 2024-08-20T06:40:26.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8fdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.182 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f41a0 0x5619fa8fd5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/1598581095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f3880 0x5619fa8f78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.177+0000 7ffa9d22e740 1 -- 172.21.15.107:0/1598581095 >> 172.21.15.107:0/1598581095 conn(0x5619fa8f16b0 msgr2=0x5619fa8feef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.178+0000 7ffa9d22e740 1 -- 172.21.15.107:0/1598581095 shutdown_connections 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.178+0000 7ffa9d22e740 1 -- 172.21.15.107:0/1598581095 wait complete. 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.178+0000 7ffa9d22e740 1 -- start start 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.178+0000 7ffa9d22e740 1 Processor -- start 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa9085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b2d00 con 0x5619fa900130 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:45662/0 (socket says 172.21.15.107:45662) 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b4170 con 0x5619fa8f3880 2024-08-20T06:40:26.183 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 -- 172.21.15.107:0/162238223 learned_addr learned my addr 172.21.15.107:0/162238223 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941b33d0 con 0x5619fa8f41a0 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa9d22e740 10 client.0 _ll_register_callbacks cb 0x7ffa800045a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa93fff640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa9085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa993d2640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 -- 172.21.15.107:0/162238223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 msgr2=0x5619fa9085b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa9085b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 -- 172.21.15.107:0/162238223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 msgr2=0x5619fa8f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa98bd1640 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619fa8faf60 con 0x5619fa8f3880 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.179+0000 7ffa993d2640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:40:26.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa98bd1640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5619fa912c30 tx=0x7ffa88012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a889ce993d376d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa73fff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88004460 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ffa88020940 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5619fa868320 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ffa88020b40 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa9d22e740 10 client.25451 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.180+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5619fa97a1a0 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5619fa868320 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ffa88027430 con 0x5619fa8f3880 2024-08-20T06:40:26.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 10 client.25451 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 10 client.25451 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 20 client.25451 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa97a990 0x5619fa97cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 4 client.25451 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5619fa865a60 con 0x5619fa97a990 2024-08-20T06:40:26.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.181+0000 7ffa93fff640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa97a990 0x5619fa97cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:26.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa73fff640 10 client.25451 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:26.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa93fff640 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa97a990 0x5619fa97cdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=306fe19ec0cc49f2 server_cookie=39e79e03c2d077da in_seq=0 out_seq=0 2024-08-20T06:40:26.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa73fff640 1 -- 172.21.15.107:0/162238223 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5619fa865a60 con 0x5619fa97a990 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa73fff640 10 client.25451 handle_command_reply: tid=0 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 client.25451 shutdown 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 msgr2=0x5619fa908070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x5619fa912c30 tx=0x7ffa88012b60 comp rx=0 tx=0).stop 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 shutdown_connections 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5619fa900130 0x5619fa8f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5619fa97a990 0x5619fa97cdb0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5619fa8f41a0 0x5619fa9085b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 --2- 172.21.15.107:0/162238223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5619fa8f3880 0x5619fa908070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 >> 172.21.15.107:0/162238223 conn(0x5619fa8f16b0 msgr2=0x5619fa8eff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.182+0000 7ffa73fff640 0 client.25451 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.183+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 shutdown_connections 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.183+0000 7ffa9d22e740 1 -- 172.21.15.107:0/162238223 wait complete. 2024-08-20T06:40:26.188 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.183+0000 7ffa9d22e740 20 client.25451 trim_cache size 0 max 16384 2024-08-20T06:40:26.189 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:40:26.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 msgr2=0x7ffa840b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 0x7ffa840b6090 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c01a000 tx=0x7ffa8c0114c0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 msgr2=0x7ffa9419fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ffa8002a910 tx=0x7ffa80003b90 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 shutdown_connections 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa94106bd0 0x7ffa941a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ffa840b3bd0 0x7ffa840b6090 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffa941020b0 0x7ffa9419fa10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 --2- 172.21.15.107:0/244890223 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ffa94101790 0x7ffa9419f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 >> 172.21.15.107:0/244890223 conn(0x7ffa940fd400 msgr2=0x7ffa941044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 shutdown_connections 2024-08-20T06:40:26.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:26.184+0000 7ffa9ae35640 1 -- 172.21.15.107:0/244890223 wait complete. 2024-08-20T06:40:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:27 smithi107 ceph-mon[18495]: pgmap v435: 97 pgs: 97 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-08-20T06:40:27.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:27 smithi183 ceph-mon[22204]: pgmap v435: 97 pgs: 97 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-08-20T06:40:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:27 smithi193 ceph-mon[21944]: pgmap v435: 97 pgs: 97 active+clean; 220 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 132 op/s 2024-08-20T06:40:28.033 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:40:28.033 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:40:28.389 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:29.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 -- 172.21.15.107:0/4020821041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff248101990 msgr2=0x7ff248101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 --2- 172.21.15.107:0/4020821041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff248101990 0x7ff248101d70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ff238009900 tx=0x7ff23802a410 comp rx=0 tx=0).stop 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 -- 172.21.15.107:0/4020821041 shutdown_connections 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 --2- 172.21.15.107:0/4020821041 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff24810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 --2- 172.21.15.107:0/4020821041 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff2481073a0 0x7ff2481077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 --2- 172.21.15.107:0/4020821041 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff248101990 0x7ff248101d70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 -- 172.21.15.107:0/4020821041 >> 172.21.15.107:0/4020821041 conn(0x7ff2480fd400 msgr2=0x7ff2480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 -- 172.21.15.107:0/4020821041 shutdown_connections 2024-08-20T06:40:29.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.054+0000 7ff24f728640 1 -- 172.21.15.107:0/4020821041 wait complete. 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.055+0000 7ff24f728640 1 Processor -- start 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.055+0000 7ff24f728640 1 -- start start 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff2481a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24819fc00 0x7ff2481a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110890 con 0x7ff2481073a0 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff24806b680 con 0x7ff24819fc00 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24f728640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff248110a10 con 0x7ff248107d20 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45346/0 (socket says 172.21.15.107:45346) 2024-08-20T06:40:29.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 -- 172.21.15.107:0/1325686079 learned_addr learned my addr 172.21.15.107:0/1325686079 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff2481a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24819fc00 0x7ff2481a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 -- 172.21.15.107:0/1325686079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 msgr2=0x7ff2481a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff2481a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 -- 172.21.15.107:0/1325686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24819fc00 msgr2=0x7ff2481a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24819fc00 0x7ff2481a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.056+0000 7ff24d4c4640 1 -- 172.21.15.107:0/1325686079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2481a0790 con 0x7ff2481073a0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff2481a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff2380057c0 tx=0x7ff238003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff23e7fc640 1 -- 172.21.15.107:0/1325686079 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23803f020 con 0x7ff2481073a0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2481a0a20 con 0x7ff2481073a0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2481a8100 con 0x7ff2481073a0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff23e7fc640 1 -- 172.21.15.107:0/1325686079 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff23803bb00 con 0x7ff2481073a0 2024-08-20T06:40:29.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.057+0000 7ff23e7fc640 1 -- 172.21.15.107:0/1325686079 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238035070 con 0x7ff2481073a0 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.060+0000 7ff251b21740 1 Processor -- start 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 -- start start 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff23e7fc640 1 -- 172.21.15.107:0/1325686079 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff238042020 con 0x7ff2481073a0 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd32790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd41050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56311cca8940 con 0x56311cd323b0 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56311cc97e40 con 0x56311cd3ec60 2024-08-20T06:40:29.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff251b21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56311cc9be80 con 0x56311cd32cd0 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24dcc5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd32790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24dcc5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd32790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45352/0 (socket says 172.21.15.107:45352) 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24dcc5640 1 -- 172.21.15.107:0/1498029673 learned_addr learned my addr 172.21.15.107:0/1498029673 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd41050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24ccc3640 1 -- 172.21.15.107:0/1498029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 msgr2=0x56311cd3e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.061+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd3e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff24ccc3640 1 -- 172.21.15.107:0/1498029673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 msgr2=0x56311cd32790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd32790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff24ccc3640 1 -- 172.21.15.107:0/1498029673 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311cc34520 con 0x56311cd3ec60 2024-08-20T06:40:29.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd41050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fa0 tx=0x7ff2440118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1988a1d54c4f640f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff2337fe640 1 -- 172.21.15.107:0/1498029673 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244004030 con 0x56311cd3ec60 2024-08-20T06:40:29.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff2337fe640 1 -- 172.21.15.107:0/1498029673 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2440035f0 con 0x56311cd3ec60 2024-08-20T06:40:29.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff2337fe640 1 -- 172.21.15.107:0/1498029673 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24400f7e0 con 0x56311cd3ec60 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.062+0000 7ff23e7fc640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 0x7ff2340b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 0x7ff2340b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff23e7fc640 1 -- 172.21.15.107:0/1325686079 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff238103bb0 con 0x7ff2481073a0 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 -- 172.21.15.107:0/1498029673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 msgr2=0x56311cd41050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd41050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fa0 tx=0x7ff2440118d0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 -- 172.21.15.107:0/1498029673 shutdown_connections 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd41050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.068 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 --2- 172.21.15.107:0/1498029673 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd32790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 -- 172.21.15.107:0/1498029673 >> 172.21.15.107:0/1498029673 conn(0x56311cd301e0 msgr2=0x56311cd31380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 0x7ff2340b6110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff240011630 tx=0x7ff240009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.063+0000 7ff251b21740 1 -- 172.21.15.107:0/1498029673 shutdown_connections 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.064+0000 7ff251b21740 1 -- 172.21.15.107:0/1498029673 wait complete. 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.064+0000 7ff251b21740 1 -- start start 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.064+0000 7ff251b21740 1 Processor -- start 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.064+0000 7ff251b21740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd3c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd3ec60 0x56311cd3c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56311cafa890 con 0x56311cd32cd0 2024-08-20T06:40:29.069 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56311cc1e700 con 0x56311cd323b0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56311cd419b0 con 0x56311cd3ec60 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff251b21740 10 client.0 _ll_register_callbacks cb 0x56311cc936f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:45694/0 (socket says 172.21.15.107:45694) 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 -- 172.21.15.107:0/2963808229 learned_addr learned my addr 172.21.15.107:0/2963808229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd3c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd3ec60 0x56311cd3c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 -- 172.21.15.107:0/2963808229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd3ec60 msgr2=0x56311cd3c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd3ec60 0x56311cd3c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 -- 172.21.15.107:0/2963808229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 msgr2=0x56311cd3c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd3c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 -- 172.21.15.107:0/2963808229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311c9f7930 con 0x56311cd323b0 2024-08-20T06:40:29.070 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd3c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2380caa90 tx=0x7ff238009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e136c3a6238e96 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.065+0000 7ff230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380c6290 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff2380c70f0 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2440035f0 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2380ca610 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff251b21740 10 client.25463 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56311cd4d570 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.066+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380cb200 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff2380e4020 con 0x56311cd323b0 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 10 client.25463 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 10 client.25463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 20 client.25463 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:29.071 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cd50d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 4 client.25463 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff2440035f0 con 0x56311cd4e8f0 2024-08-20T06:40:29.072 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cd50d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff230ff9640 10 client.25463 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.067+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cd50d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d2139144c0e9d2 server_cookie=f4d41c0ecfaf969b in_seq=0 out_seq=0 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.068+0000 7ff230ff9640 1 -- 172.21.15.107:0/2963808229 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7ff2440035f0 con 0x56311cd4e8f0 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.068+0000 7ff230ff9640 10 client.25463 handle_command_reply: tid=0 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.068+0000 7ff251b21740 1 client.25463 shutdown 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.068+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 msgr2=0x56311cd3bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.068+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff2380caa90 tx=0x7ff238009260 comp rx=0 tx=0).stop 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 shutdown_connections 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cd50d70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd3ec60 0x56311cd3c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd3c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.073 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff230ff9640 0 client.25463 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 --2- 172.21.15.107:0/2963808229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd323b0 0x56311cd3bc20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 >> 172.21.15.107:0/2963808229 conn(0x56311cd301e0 msgr2=0x56311cd35d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 shutdown_connections 2024-08-20T06:40:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 1 -- 172.21.15.107:0/2963808229 wait complete. 2024-08-20T06:40:29.074 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.069+0000 7ff251b21740 20 client.25463 trim_cache size 0 max 16384 2024-08-20T06:40:29.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff251b21740 1 Processor -- start 2024-08-20T06:40:29.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff251b21740 1 -- start start 2024-08-20T06:40:29.104 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff251b21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff24d4c4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45370/0 (socket says 172.21.15.107:45370) 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.103+0000 7ff251b21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd56680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 -- 172.21.15.107:0/3143392056 learned_addr learned my addr 172.21.15.107:0/3143392056 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff251b21740 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481b3070 con 0x56311cd323b0 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481b0940 con 0x56311cd3ec60 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481ae770 con 0x56311cd32cd0 2024-08-20T06:40:29.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd56680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 -- 172.21.15.107:0/3143392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 msgr2=0x56311cd56680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd56680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 -- 172.21.15.107:0/3143392056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 msgr2=0x56311cd56bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd56bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 -- 172.21.15.107:0/3143392056 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56311cd30010 con 0x56311cd323b0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2380c6b80 tx=0x7ff238005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bc06ff09f14b57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.104+0000 7ff232ffd640 1 -- 172.21.15.107:0/3143392056 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380c6290 con 0x56311cd323b0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff232ffd640 1 -- 172.21.15.107:0/3143392056 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380c6f20 con 0x56311cd323b0 2024-08-20T06:40:29.107 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff232ffd640 1 -- 172.21.15.107:0/3143392056 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380b20f0 con 0x56311cd323b0 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 msgr2=0x56311cd36420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff2380c6b80 tx=0x7ff238005c90 comp rx=0 tx=0).stop 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 shutdown_connections 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd56bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd32cd0 0x56311cd56680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 --2- 172.21.15.107:0/3143392056 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd323b0 0x56311cd36420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 >> 172.21.15.107:0/3143392056 conn(0x56311cd301e0 msgr2=0x56311cd3da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.105+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 shutdown_connections 2024-08-20T06:40:29.111 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.106+0000 7ff251b21740 1 -- 172.21.15.107:0/3143392056 wait complete. 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.106+0000 7ff251b21740 1 -- start start 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.106+0000 7ff251b21740 1 Processor -- start 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.106+0000 7ff251b21740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd323b0 0x56311cd42c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd43710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481aee00 con 0x56311cd32cd0 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481b0270 con 0x56311cd3ec60 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff2481af4d0 con 0x56311cd323b0 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff251b21740 10 client.0 _ll_register_callbacks cb 0x7ff238005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24ccc3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24ccc3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45382/0 (socket says 172.21.15.107:45382) 2024-08-20T06:40:29.112 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24ccc3640 1 -- 172.21.15.107:0/2252942414 learned_addr learned my addr 172.21.15.107:0/2252942414 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd43710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd323b0 0x56311cd42c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 -- 172.21.15.107:0/2252942414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd323b0 msgr2=0x56311cd42c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd323b0 0x56311cd42c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 -- 172.21.15.107:0/2252942414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 msgr2=0x56311cd431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 -- 172.21.15.107:0/2252942414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56311cdb88a0 con 0x56311cd3ec60 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24ccc3640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff24dcc5640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd43710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff240005fd0 tx=0x7ff240004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc341470cd872e07 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.107+0000 7ff230ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24002e060 con 0x56311cd3ec60 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff24000f770 con 0x56311cd3ec60 2024-08-20T06:40:29.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2380c6f20 con 0x56311cd3ec60 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff240004820 con 0x56311cd3ec60 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 10 client.25469 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56311cdb9c40 con 0x56311cd3ec60 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff2380c6f20 con 0x56311cd3ec60 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff240028070 con 0x56311cd3ec60 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 10 client.25469 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 20 client.25469 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.108+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cdbc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.109+0000 7ff251b21740 4 client.25469 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.109+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56311ccaa890 con 0x56311cd4e8f0 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.109+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cdbc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.109+0000 7ff230ff9640 10 client.25469 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:29.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.109+0000 7ff24d4c4640 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cdbc470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=487157e2f0bde186 server_cookie=c64f7a46fe8f096e in_seq=0 out_seq=0 2024-08-20T06:40:29.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.110+0000 7ff230ff9640 1 -- 172.21.15.107:0/2252942414 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56311ccaa890 con 0x56311cd4e8f0 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.110+0000 7ff230ff9640 10 client.25469 handle_command_reply: tid=0 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.110+0000 7ff251b21740 1 client.25469 shutdown 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 msgr2=0x56311cd43710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd43710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff240005fd0 tx=0x7ff240004a40 comp rx=0 tx=0).stop 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 shutdown_connections 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56311cd3ec60 0x56311cd43710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56311cd32cd0 0x56311cd431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56311cd4e8f0 0x56311cdbc470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 --2- 172.21.15.107:0/2252942414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56311cd323b0 0x56311cd42c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 >> 172.21.15.107:0/2252942414 conn(0x56311cd301e0 msgr2=0x56311cd49ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff230ff9640 0 client.25469 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 shutdown_connections 2024-08-20T06:40:29.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 1 -- 172.21.15.107:0/2252942414 wait complete. 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.111+0000 7ff251b21740 20 client.25469 trim_cache size 0 max 16384 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "d659d6ed-14ac-47c3-91c3-5b33a7dee035", 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:40:29.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 msgr2=0x7ff2340b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 0x7ff2340b6110 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff240011630 tx=0x7ff240009500 comp rx=0 tx=0).stop 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 msgr2=0x7ff2481a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff2380057c0 tx=0x7ff238003c10 comp rx=0 tx=0).stop 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 shutdown_connections 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff24819fc00 0x7ff2481a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff2340b3c50 0x7ff2340b6110 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff248107d20 0x7ff2481a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 --2- 172.21.15.107:0/1325686079 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff2481073a0 0x7ff2481a0e00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:29.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 >> 172.21.15.107:0/1325686079 conn(0x7ff2480fd400 msgr2=0x7ff2480fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:29.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 shutdown_connections 2024-08-20T06:40:29.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:29.112+0000 7ff24f728640 1 -- 172.21.15.107:0/1325686079 wait complete. 2024-08-20T06:40:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:29 smithi107 ceph-mon[18495]: pgmap v436: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-08-20T06:40:29.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:29 smithi183 ceph-mon[22204]: pgmap v436: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-08-20T06:40:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:29 smithi193 ceph-mon[21944]: pgmap v436: 97 pgs: 97 active+clean; 227 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.9 MiB/s wr, 147 op/s 2024-08-20T06:40:29.953 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:40:30.318 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:40:30.444 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:30 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:40:30.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:30 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:40:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:30 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 -- 172.21.15.107:0/3747534943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae741039a0 msgr2=0x7fae74103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 --2- 172.21.15.107:0/3747534943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae741039a0 0x7fae74103d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fae64009900 tx=0x7fae6402a410 comp rx=0 tx=0).stop 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 -- 172.21.15.107:0/3747534943 shutdown_connections 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 --2- 172.21.15.107:0/3747534943 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae7410ce00 0x7fae7410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 --2- 172.21.15.107:0/3747534943 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae7410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 --2- 172.21.15.107:0/3747534943 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae741039a0 0x7fae74103d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 -- 172.21.15.107:0/3747534943 >> 172.21.15.107:0/3747534943 conn(0x7fae740fd400 msgr2=0x7fae740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 -- 172.21.15.107:0/3747534943 shutdown_connections 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.861+0000 7fae79cd7640 1 -- 172.21.15.107:0/3747534943 wait complete. 2024-08-20T06:40:30.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.862+0000 7fae79cd7640 1 Processor -- start 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.862+0000 7fae79cd7640 1 -- start start 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 0x7fae74198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 0x7fae7419e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae74112920 con 0x7fae7410ce00 2024-08-20T06:40:30.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae741127a0 con 0x7fae741042c0 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae73fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 0x7fae7419e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae79cd7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae74112aa0 con 0x7fae741039a0 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:45718/0 (socket says 172.21.15.107:45718) 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae72ffd640 1 -- 172.21.15.107:0/1666113415 learned_addr learned my addr 172.21.15.107:0/1666113415 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:30.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.863+0000 7fae737fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 0x7fae74198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 -- 172.21.15.107:0/1666113415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 msgr2=0x7fae74198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 0x7fae74198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 -- 172.21.15.107:0/1666113415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 msgr2=0x7fae741991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 -- 172.21.15.107:0/1666113415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7419ed40 con 0x7fae7410ce00 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae737fe640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 0x7fae74198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae72ffd640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae73fff640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 0x7fae7419e570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009fd0 tx=0x7fae6c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae70ff9640 1 -- 172.21.15.107:0/1666113415 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c00ce20 con 0x7fae7410ce00 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae70ff9640 1 -- 172.21.15.107:0/1666113415 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae6c005020 con 0x7fae7410ce00 2024-08-20T06:40:30.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae70ff9640 1 -- 172.21.15.107:0/1666113415 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c010690 con 0x7fae7410ce00 2024-08-20T06:40:30.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae7419f030 con 0x7fae7410ce00 2024-08-20T06:40:30.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.864+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae741a56f0 con 0x7fae7410ce00 2024-08-20T06:40:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 Processor -- start 2024-08-20T06:40:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae70ff9640 1 -- 172.21.15.107:0/1666113415 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fae6c004a40 con 0x7fae7410ce00 2024-08-20T06:40:30.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 -- start start 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ee5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7e578e0 con 0x55e1c7ef5790 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7e46de0 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.868+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7e4ae20 con 0x55e1c7ee4c40 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ee5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:45728/0 (socket says 172.21.15.107:45728) 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 -- 172.21.15.107:0/3774296237 learned_addr learned my addr 172.21.15.107:0/3774296237 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae72ffd640 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 -- 172.21.15.107:0/3774296237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee4c40 msgr2=0x55e1c7ee5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ee5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae737fe640 1 -- 172.21.15.107:0/3774296237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 msgr2=0x55e1c7ef7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.869+0000 7fae70ff9640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 0x7fae5c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae737fe640 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae737fe640 1 -- 172.21.15.107:0/3774296237 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c7ba6650 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae72ffd640 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae70ff9640 1 -- 172.21.15.107:0/1666113415 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fae6c0e4210 con 0x7fae7410ce00 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae72ffd640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 0x7fae5c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae737fe640 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae64033000 tx=0x7fae64000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7490af5abb90011e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae59ffb640 1 -- 172.21.15.107:0/3774296237 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64040070 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae59ffb640 1 -- 172.21.15.107:0/3774296237 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae64032e30 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.870+0000 7fae59ffb640 1 -- 172.21.15.107:0/3774296237 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae64035040 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.871+0000 7fae7c0d0740 1 -- 172.21.15.107:0/3774296237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 msgr2=0x55e1c7ee5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.871+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fae64033000 tx=0x7fae64000bc0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.871+0000 7fae72ffd640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 0x7fae5c0b6110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fae68009fd0 tx=0x7fae680046c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 -- 172.21.15.107:0/3774296237 shutdown_connections 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5560 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/3774296237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ee5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 -- 172.21.15.107:0/3774296237 >> 172.21.15.107:0/3774296237 conn(0x55e1c7ee2a70 msgr2=0x55e1c7ee3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 -- 172.21.15.107:0/3774296237 shutdown_connections 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 -- 172.21.15.107:0/3774296237 wait complete. 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 -- start start 2024-08-20T06:40:30.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.872+0000 7fae7c0d0740 1 Processor -- start 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7edef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7edf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7ca9700 con 0x55e1c7ee4c40 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7dcd7c0 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e1c7ef8570 con 0x55e1c7ef5790 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae7c0d0740 10 client.0 _ll_register_callbacks cb 0x55e1c7e406b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45410/0 (socket says 172.21.15.107:45410) 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 -- 172.21.15.107:0/2569958775 learned_addr learned my addr 172.21.15.107:0/2569958775 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:30.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae737fe640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7edef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae73fff640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7edf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 -- 172.21.15.107:0/2569958775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 msgr2=0x55e1c7edf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7edf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 -- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 msgr2=0x55e1c7edef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7edef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 -- 172.21.15.107:0/2569958775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c7f01900 con 0x55e1c7ee4c40 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae73fff640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7edf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae68011eb0 tx=0x7fae68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb25e7ad7e858dcf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.873+0000 7fae52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6802e060 con 0x55e1c7ee4c40 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fae6801a070 con 0x55e1c7ee4c40 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1c7df6100 con 0x55e1c7ee4c40 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae68010e80 con 0x55e1c7ee4c40 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae7c0d0740 10 client.15966 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:30.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1c7f04290 con 0x55e1c7ee4c40 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e1c7df6100 con 0x55e1c7ee4c40 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.874+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae68003c00 con 0x55e1c7ee4c40 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 10 client.15966 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 10 client.15966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 20 client.15966 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f057a0 0x55e1c7f07bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 4 client.15966 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c7e59830 con 0x55e1c7f057a0 2024-08-20T06:40:30.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.875+0000 7fae737fe640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f057a0 0x55e1c7f07bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.876+0000 7fae737fe640 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f057a0 0x55e1c7f07bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3615846c24fc45fe server_cookie=1b5e7919414a7f7 in_seq=0 out_seq=0 2024-08-20T06:40:30.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.876+0000 7fae52ffd640 10 client.15966 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae52ffd640 1 -- 172.21.15.107:0/2569958775 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1c7e59830 con 0x55e1c7f057a0 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae52ffd640 10 client.15966 handle_command_reply: tid=0 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 client.15966 shutdown 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 msgr2=0x55e1c7edf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fae68011eb0 tx=0x7fae68004730 comp rx=0 tx=0).stop 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 shutdown_connections 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f057a0 0x55e1c7f07bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7edf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee4c40 0x55e1c7edf450 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae52ffd640 0 client.15966 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2569958775 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7edef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.877+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 >> 172.21.15.107:0/2569958775 conn(0x55e1c7ee2a70 msgr2=0x55e1c7ee7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.878+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 shutdown_connections 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.878+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2569958775 wait complete. 2024-08-20T06:40:30.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.878+0000 7fae7c0d0740 20 client.15966 trim_cache size 0 max 16384 2024-08-20T06:40:30.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 Processor -- start 2024-08-20T06:40:30.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 -- start start 2024-08-20T06:40:30.901 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ef7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae741ae6e0 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae741abfb0 con 0x55e1c7ee4c40 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae741a9de0 con 0x55e1c7ef5790 2024-08-20T06:40:30.902 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.900+0000 7fae737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae737fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45418/0 (socket says 172.21.15.107:45418) 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae737fe640 1 -- 172.21.15.107:0/2623389374 learned_addr learned my addr 172.21.15.107:0/2623389374 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ef7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 -- 172.21.15.107:0/2623389374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 msgr2=0x55e1c7ef7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ef7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 -- 172.21.15.107:0/2623389374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 msgr2=0x55e1c7ee5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.903 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 -- 172.21.15.107:0/2623389374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c7ee3750 con 0x55e1c7ef5790 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae73fff640 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0ab0f0 tx=0x7fae6c0aad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed6f75b2ab7622c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae597fa640 1 -- 172.21.15.107:0/2623389374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0c4070 con 0x55e1c7ef5790 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae597fa640 1 -- 172.21.15.107:0/2623389374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae6c0af760 con 0x55e1c7ef5790 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.901+0000 7fae597fa640 1 -- 172.21.15.107:0/2623389374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c0b3d80 con 0x55e1c7ef5790 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2623389374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 msgr2=0x55e1c7ef7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7840 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0ab0f0 tx=0x7fae6c0aad80 comp rx=0 tx=0).stop 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2623389374 shutdown_connections 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ef5790 0x55e1c7ef7840 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7ef7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2623389374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7ee5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2623389374 >> 172.21.15.107:0/2623389374 conn(0x55e1c7ee2a70 msgr2=0x55e1c7ee5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2623389374 shutdown_connections 2024-08-20T06:40:30.904 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.902+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2623389374 wait complete. 2024-08-20T06:40:30.906 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.903+0000 7fae7c0d0740 1 -- start start 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.903+0000 7fae7c0d0740 1 Processor -- start 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.903+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7efda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7efdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fae741aa470 con 0x55e1c7ef5790 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fae741ab8e0 con 0x55e1c7ee4c40 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fae741aab40 con 0x55e1c7ee1fe0 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45424/0 (socket says 172.21.15.107:45424) 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 -- 172.21.15.107:0/2253168080 learned_addr learned my addr 172.21.15.107:0/2253168080 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:40:30.907 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7efdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae737fe640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7efda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae7c0d0740 10 client.0 _ll_register_callbacks cb 0x7fae68011430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 -- 172.21.15.107:0/2253168080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 msgr2=0x55e1c7efda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7efda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 -- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 msgr2=0x55e1c7efdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7efdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae73fff640 1 -- 172.21.15.107:0/2253168080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c7f0d030 con 0x55e1c7ef5790 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae737fe640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7efda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.904+0000 7fae72ffd640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7efdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae73fff640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0a7190 tx=0x7fae6c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9fc08723b6d603 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6c08a060 con 0x55e1c7ef5790 2024-08-20T06:40:30.908 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fae6c0a7750 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae6c0af760 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fae6c0c4040 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae7c0d0740 10 client.15972 fetch_fsmap learned FSMap version 27 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e1c7f6fe80 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.905+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fae6c0af760 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fae6c0a6ad0 con 0x55e1c7ef5790 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 10 client.15972 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 20 client.15972 populate_metadata read hostname 'smithi107' 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f70570 0x55e1c7f72990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 4 client.15972 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1c7e552d0 con 0x55e1c7f70570 2024-08-20T06:40:30.909 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae737fe640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f70570 0x55e1c7f72990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.906+0000 7fae52ffd640 10 client.15972 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.907+0000 7fae737fe640 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f70570 0x55e1c7f72990 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f9e19258a99474 server_cookie=357044ce65aa494e in_seq=0 out_seq=0 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.908+0000 7fae52ffd640 1 -- 172.21.15.107:0/2253168080 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+266 (crc 0 0 0) 0x55e1c7e552d0 con 0x55e1c7f70570 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.908+0000 7fae52ffd640 10 client.15972 handle_command_reply: tid=0 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.908+0000 7fae7c0d0740 1 client.15972 shutdown 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.908+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 msgr2=0x55e1c7f0c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.908+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fae6c0a7190 tx=0x7fae6c005700 comp rx=0 tx=0).stop 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 shutdown_connections 2024-08-20T06:40:30.911 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e1c7ef5790 0x55e1c7f0c920 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e1c7ee4c40 0x55e1c7efdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e1c7f70570 0x55e1c7f72990 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 --2- 172.21.15.107:0/2253168080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e1c7ee1fe0 0x55e1c7efda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 >> 172.21.15.107:0/2253168080 conn(0x55e1c7ee2a70 msgr2=0x55e1c7ee7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae52ffd640 0 client.15972 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 shutdown_connections 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 1 -- 172.21.15.107:0/2253168080 wait complete. 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.909+0000 7fae7c0d0740 20 client.15972 trim_cache size 0 max 16384 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (76 inodes in the stack)", 2024-08-20T06:40:30.912 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: "d659d6ed-14ac-47c3-91c3-5b33a7dee035": { 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "d659d6ed-14ac-47c3-91c3-5b33a7dee035", 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:40:30.913 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 msgr2=0x7fae5c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 0x7fae5c0b6110 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fae68009fd0 tx=0x7fae680046c0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 msgr2=0x7fae7419e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 0x7fae7419e570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fae6c009fd0 tx=0x7fae6c00ef10 comp rx=0 tx=0).stop 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 shutdown_connections 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fae7410ce00 0x7fae7419e570 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.914 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fae741042c0 0x7fae741991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fae5c0b3c50 0x7fae5c0b6110 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 --2- 172.21.15.107:0/1666113415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae741039a0 0x7fae74198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:40:30.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 >> 172.21.15.107:0/1666113415 conn(0x7fae740fd400 msgr2=0x7fae74104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:40:30.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 shutdown_connections 2024-08-20T06:40:30.915 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:40:30.910+0000 7fae79cd7640 1 -- 172.21.15.107:0/1666113415 wait complete. 2024-08-20T06:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:31 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:31 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:31 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:31 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:31 smithi183 ceph-mon[22204]: pgmap v437: 97 pgs: 97 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 140 op/s 2024-08-20T06:40:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:31 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:40:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:31 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:31 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:40:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:31 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:31 smithi193 ceph-mon[21944]: pgmap v437: 97 pgs: 97 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 140 op/s 2024-08-20T06:40:31.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:31 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:40:31.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:31 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:31 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:40:31.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:31 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:31.601 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:31 smithi107 ceph-mon[18495]: pgmap v437: 97 pgs: 97 active+clean; 229 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.5 MiB/s wr, 140 op/s 2024-08-20T06:40:31.717 INFO:tasks.cephfs.filesystem:scrub status for tag:d659d6ed-14ac-47c3-91c3-5b33a7dee035 - {'path': '/', 'tag': 'd659d6ed-14ac-47c3-91c3-5b33a7dee035', 'options': 'recursive,force'} 2024-08-20T06:40:32.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:32 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:32.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:32 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:32.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:32 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:40:33.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:33 smithi183 ceph-mon[22204]: pgmap v438: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-08-20T06:40:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:33 smithi193 ceph-mon[21944]: pgmap v438: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-08-20T06:40:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:33 smithi107 ceph-mon[18495]: pgmap v438: 97 pgs: 97 active+clean; 235 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 153 op/s 2024-08-20T06:40:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:35 smithi183 ceph-mon[22204]: pgmap v439: 97 pgs: 97 active+clean; 241 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-08-20T06:40:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:35 smithi193 ceph-mon[21944]: pgmap v439: 97 pgs: 97 active+clean; 241 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-08-20T06:40:35.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:35 smithi107 ceph-mon[18495]: pgmap v439: 97 pgs: 97 active+clean; 241 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 151 op/s 2024-08-20T06:40:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: 49 13709 164 9245 178 3853 252 2024-08-20T06:40:36.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:36 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:40:36.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:36 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:40:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:36 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:40:36.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:36 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:40:36.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:36 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:40:36.631 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:36 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:40:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:37 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:40:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:37 smithi183 ceph-mon[22204]: pgmap v440: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-08-20T06:40:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:37 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:40:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:37 smithi193 ceph-mon[21944]: pgmap v440: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-08-20T06:40:37.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:37 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:40:37.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:37 smithi107 ceph-mon[18495]: pgmap v440: 97 pgs: 97 active+clean; 242 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 126 op/s 2024-08-20T06:40:39.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:39 smithi183 ceph-mon[22204]: pgmap v441: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 126 op/s 2024-08-20T06:40:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:39 smithi193 ceph-mon[21944]: pgmap v441: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 126 op/s 2024-08-20T06:40:39.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:39 smithi107 ceph-mon[18495]: pgmap v441: 97 pgs: 97 active+clean; 248 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 126 op/s 2024-08-20T06:40:41.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:41 smithi183 ceph-mon[22204]: pgmap v442: 97 pgs: 97 active+clean; 255 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-20T06:40:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:41 smithi193 ceph-mon[21944]: pgmap v442: 97 pgs: 97 active+clean; 255 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-20T06:40:41.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:41 smithi107 ceph-mon[18495]: pgmap v442: 97 pgs: 97 active+clean; 255 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-08-20T06:40:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:43 smithi183 ceph-mon[22204]: pgmap v443: 97 pgs: 97 active+clean; 256 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 130 op/s 2024-08-20T06:40:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:43 smithi193 ceph-mon[21944]: pgmap v443: 97 pgs: 97 active+clean; 256 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 130 op/s 2024-08-20T06:40:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:43 smithi107 ceph-mon[18495]: pgmap v443: 97 pgs: 97 active+clean; 256 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 130 op/s 2024-08-20T06:40:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:45 smithi183 ceph-mon[22204]: pgmap v444: 97 pgs: 97 active+clean; 270 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-08-20T06:40:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:45 smithi193 ceph-mon[21944]: pgmap v444: 97 pgs: 97 active+clean; 270 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-08-20T06:40:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:45 smithi107 ceph-mon[18495]: pgmap v444: 97 pgs: 97 active+clean; 270 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.2 MiB/s wr, 151 op/s 2024-08-20T06:40:45.886 INFO:tasks.workunit.client.0.smithi107.stdout: 54 13943 201 8709 174 4066 415 2024-08-20T06:40:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:47 smithi183 ceph-mon[22204]: pgmap v445: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 133 op/s 2024-08-20T06:40:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:47 smithi193 ceph-mon[21944]: pgmap v445: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 133 op/s 2024-08-20T06:40:47.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:47 smithi107 ceph-mon[18495]: pgmap v445: 97 pgs: 97 active+clean; 271 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.7 MiB/s wr, 133 op/s 2024-08-20T06:40:49.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:49 smithi183 ceph-mon[22204]: pgmap v446: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-08-20T06:40:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:49 smithi193 ceph-mon[21944]: pgmap v446: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-08-20T06:40:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:49 smithi107 ceph-mon[18495]: pgmap v446: 97 pgs: 97 active+clean; 280 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 154 op/s 2024-08-20T06:40:51.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:51 smithi183 ceph-mon[22204]: pgmap v447: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 151 op/s 2024-08-20T06:40:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:51 smithi193 ceph-mon[21944]: pgmap v447: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 151 op/s 2024-08-20T06:40:51.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:51 smithi107 ceph-mon[18495]: pgmap v447: 97 pgs: 97 active+clean; 281 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 151 op/s 2024-08-20T06:40:53.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:53 smithi183 ceph-mon[22204]: pgmap v448: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 148 op/s 2024-08-20T06:40:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:53 smithi193 ceph-mon[21944]: pgmap v448: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 148 op/s 2024-08-20T06:40:53.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:53 smithi107 ceph-mon[18495]: pgmap v448: 97 pgs: 97 active+clean; 291 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 148 op/s 2024-08-20T06:40:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:55 smithi183 ceph-mon[22204]: pgmap v449: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-08-20T06:40:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:55 smithi193 ceph-mon[21944]: pgmap v449: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-08-20T06:40:55.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:55 smithi107 ceph-mon[18495]: pgmap v449: 97 pgs: 97 active+clean; 298 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.8 MiB/s wr, 163 op/s 2024-08-20T06:40:55.886 INFO:tasks.workunit.client.0.smithi107.stdout:write(): Bad file descriptor 58 13665 166 8819 181 3559 402 2024-08-20T06:40:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:57 smithi183 ceph-mon[22204]: pgmap v450: 97 pgs: 97 active+clean; 299 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 123 op/s 2024-08-20T06:40:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:57 smithi193 ceph-mon[21944]: pgmap v450: 97 pgs: 97 active+clean; 299 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 123 op/s 2024-08-20T06:40:57.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:57 smithi107 ceph-mon[18495]: pgmap v450: 97 pgs: 97 active+clean; 299 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.7 MiB/s wr, 123 op/s 2024-08-20T06:40:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:40:59 smithi183 ceph-mon[22204]: pgmap v451: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-08-20T06:40:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:40:59 smithi193 ceph-mon[21944]: pgmap v451: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-08-20T06:40:59.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:40:59 smithi107 ceph-mon[18495]: pgmap v451: 97 pgs: 97 active+clean; 305 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 139 op/s 2024-08-20T06:41:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:01 smithi183 ceph-mon[22204]: pgmap v452: 97 pgs: 97 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 127 op/s 2024-08-20T06:41:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:01 smithi193 ceph-mon[21944]: pgmap v452: 97 pgs: 97 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 127 op/s 2024-08-20T06:41:01.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:01 smithi107 ceph-mon[18495]: pgmap v452: 97 pgs: 97 active+clean; 312 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 127 op/s 2024-08-20T06:41:01.718 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:41:02.078 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 -- 172.21.15.107:0/2936649312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581019d0 msgr2=0x7fdd58101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/2936649312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581019d0 0x7fdd58101db0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009900 tx=0x7fdd4802a410 comp rx=0 tx=0).stop 2024-08-20T06:41:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 -- 172.21.15.107:0/2936649312 shutdown_connections 2024-08-20T06:41:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/2936649312 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd58107d60 0x7fdd5810e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/2936649312 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd58107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/2936649312 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd581019d0 0x7fdd58101db0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 -- 172.21.15.107:0/2936649312 >> 172.21.15.107:0/2936649312 conn(0x7fdd580fd460 msgr2=0x7fdd580ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 -- 172.21.15.107:0/2936649312 shutdown_connections 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.637+0000 7fdd5cdef640 1 -- 172.21.15.107:0/2936649312 wait complete. 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.638+0000 7fdd5cdef640 1 Processor -- start 2024-08-20T06:41:02.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.638+0000 7fdd5cdef640 1 -- start start 2024-08-20T06:41:02.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd581019d0 0x7fdd58110970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd5810eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58112990 con 0x7fdd58107d60 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58112810 con 0x7fdd581019d0 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5cdef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd58112b10 con 0x7fdd581073e0 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33130/0 (socket says 172.21.15.107:33130) 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 -- 172.21.15.107:0/1170725137 learned_addr learned my addr 172.21.15.107:0/1170725137 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd5810eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd5659c640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd581019d0 0x7fdd58110970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 -- 172.21.15.107:0/1170725137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 msgr2=0x7fdd5810eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd5810eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 -- 172.21.15.107:0/1170725137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd581019d0 msgr2=0x7fdd58110970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd581019d0 0x7fdd58110970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd56d9d640 1 -- 172.21.15.107:0/1170725137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd581ab840 con 0x7fdd58107d60 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.639+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd5810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000eb10 tx=0x7fdd5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd477fe640 1 -- 172.21.15.107:0/1170725137 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50004b20 con 0x7fdd58107d60 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd477fe640 1 -- 172.21.15.107:0/1170725137 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd50004cc0 con 0x7fdd58107d60 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd581aba10 con 0x7fdd58107d60 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd477fe640 1 -- 172.21.15.107:0/1170725137 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd50010690 con 0x7fdd58107d60 2024-08-20T06:41:02.643 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.640+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd581abf20 con 0x7fdd58107d60 2024-08-20T06:41:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.643+0000 7fdd5f1e8740 1 Processor -- start 2024-08-20T06:41:02.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd477fe640 1 -- 172.21.15.107:0/1170725137 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fdd50005020 con 0x7fdd58107d60 2024-08-20T06:41:02.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 -- start start 2024-08-20T06:41:02.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e900000 0x56235e90ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e90e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56235e8768e0 con 0x56235e8ff6e0 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56235e865de0 con 0x56235e900000 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.644+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56235e869e20 con 0x56235e90bf90 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33136/0 (socket says 172.21.15.107:33136) 2024-08-20T06:41:02.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 -- 172.21.15.107:0/1323439378 learned_addr learned my addr 172.21.15.107:0/1323439378 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e90e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 -- 172.21.15.107:0/1323439378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 msgr2=0x56235e90e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e90e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 -- 172.21.15.107:0/1323439378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e900000 msgr2=0x56235e90ba50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e900000 0x56235e90ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 -- 172.21.15.107:0/1323439378 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e5c5650 con 0x56235e8ff6e0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.645+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c013990 tx=0x7fdd4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25604dd415aaa4eb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd44ff9640 1 -- 172.21.15.107:0/1323439378 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c004030 con 0x56235e8ff6e0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd477fe640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 0x7fdd400b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd44ff9640 1 -- 172.21.15.107:0/1323439378 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd4c00f940 con 0x56235e8ff6e0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd44ff9640 1 -- 172.21.15.107:0/1323439378 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c005380 con 0x56235e8ff6e0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd5659c640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 0x7fdd400b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.646+0000 7fdd477fe640 1 -- 172.21.15.107:0/1170725137 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd500e4120 con 0x7fdd58107d60 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/1323439378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 msgr2=0x56235e8ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5659c640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 0x7fdd400b6110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd480057c0 tx=0x7fdd480058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c013990 tx=0x7fdd4c009c00 comp rx=0 tx=0).stop 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/1323439378 shutdown_connections 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e90e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e900000 0x56235e90ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/1323439378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e8ff6e0 0x56235e8ffac0 secure :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c013990 tx=0x7fdd4c009c00 comp rx=0 tx=0).stop 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/1323439378 >> 172.21.15.107:0/1323439378 conn(0x56235e8fd510 msgr2=0x56235e8fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/1323439378 shutdown_connections 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/1323439378 wait complete. 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 -- start start 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.647+0000 7fdd5f1e8740 1 Processor -- start 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e900000 0x56235e909180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e917fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56235e6c8700 con 0x56235e90bf90 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x56235e7ec7c0 con 0x56235e909c00 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x56235e90ec40 con 0x56235e900000 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33148/0 (socket says 172.21.15.107:33148) 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 -- 172.21.15.107:0/2331420854 learned_addr learned my addr 172.21.15.107:0/2331420854 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5f1e8740 10 client.0 _ll_register_callbacks cb 0x56235e85f6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e900000 0x56235e909180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e917fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 -- 172.21.15.107:0/2331420854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e900000 msgr2=0x56235e909180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e900000 0x56235e909180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 -- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 msgr2=0x56235e917fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.648+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e917fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd55d9b640 1 -- 172.21.15.107:0/2331420854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e918690 con 0x56235e90bf90 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c01b000 tx=0x7fdd4c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb141cfffece265 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00fdf0 con 0x56235e90bf90 2024-08-20T06:41:02.652 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd4c004030 con 0x56235e90bf90 2024-08-20T06:41:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56235e815100 con 0x56235e90bf90 2024-08-20T06:41:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd4c004230 con 0x56235e90bf90 2024-08-20T06:41:02.653 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd5f1e8740 10 client.15984 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:02.654 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.649+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56235e91afe0 con 0x56235e90bf90 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56235e815100 con 0x56235e90bf90 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd4c01d020 con 0x56235e90bf90 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 10 client.15984 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 20 client.15984 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e91c6e0 0x56235e91ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 4 client.15984 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56235e878830 con 0x56235e91c6e0 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.650+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e91c6e0 0x56235e91ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.651+0000 7fdd3dffb640 10 client.15984 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.651+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e91c6e0 0x56235e91ead0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bc49254fdecc5a server_cookie=231b202a074d7173 in_seq=0 out_seq=0 2024-08-20T06:41:02.655 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.651+0000 7fdd3dffb640 1 -- 172.21.15.107:0/2331420854 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56235e878830 con 0x56235e91c6e0 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.651+0000 7fdd3dffb640 10 client.15984 handle_command_reply: tid=0 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.651+0000 7fdd5f1e8740 1 client.15984 shutdown 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 msgr2=0x56235e9096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c01b000 tx=0x7fdd4c005dc0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 shutdown_connections 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e917fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e90bf90 0x56235e9096c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e91c6e0 0x56235e91ead0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2331420854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e900000 0x56235e909180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 >> 172.21.15.107:0/2331420854 conn(0x56235e8fd510 msgr2=0x56235e903080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd3dffb640 0 client.15984 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 shutdown_connections 2024-08-20T06:41:02.656 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2331420854 wait complete. 2024-08-20T06:41:02.657 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.652+0000 7fdd5f1e8740 20 client.15984 trim_cache size 0 max 16384 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.678+0000 7fdd5f1e8740 1 Processor -- start 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 -- start start 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e909c00 0x56235e8fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e90bf90 0x56235e8fc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b6ee0 con 0x56235e900000 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b47b0 con 0x56235e90bf90 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5f1e8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b25e0 con 0x56235e909c00 2024-08-20T06:41:02.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33150/0 (socket says 172.21.15.107:33150) 2024-08-20T06:41:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 -- 172.21.15.107:0/2697523219 learned_addr learned my addr 172.21.15.107:0/2697523219 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e90bf90 0x56235e8fc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e909c00 0x56235e8fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 -- 172.21.15.107:0/2697523219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e909c00 msgr2=0x56235e8fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e909c00 0x56235e8fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 -- 172.21.15.107:0/2697523219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e90bf90 msgr2=0x56235e8fc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e90bf90 0x56235e8fc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.679+0000 7fdd5659c640 1 -- 172.21.15.107:0/2697523219 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56235e8fdf90 con 0x56235e900000 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.680+0000 7fdd5659c640 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000c00 tx=0x7fdd4803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf83a39bb8cf185 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.680+0000 7fdd3ffff640 1 -- 172.21.15.107:0/2697523219 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48035050 con 0x56235e900000 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.680+0000 7fdd3ffff640 1 -- 172.21.15.107:0/2697523219 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd48004300 con 0x56235e900000 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.680+0000 7fdd3ffff640 1 -- 172.21.15.107:0/2697523219 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48053590 con 0x56235e900000 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2697523219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 msgr2=0x56235e917f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000c00 tx=0x7fdd4803e000 comp rx=0 tx=0).stop 2024-08-20T06:41:02.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2697523219 shutdown_connections 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e90bf90 0x56235e8fc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e909c00 0x56235e8fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/2697523219 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e917f10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2697523219 >> 172.21.15.107:0/2697523219 conn(0x56235e8fd510 msgr2=0x56235e8fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2697523219 shutdown_connections 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/2697523219 wait complete. 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 -- start start 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.681+0000 7fdd5f1e8740 1 Processor -- start 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33156/0 (socket says 172.21.15.107:33156) 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e9205c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 -- 172.21.15.107:0/818990865 learned_addr learned my addr 172.21.15.107:0/818990865 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e97cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b2c70 con 0x56235e900000 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b40e0 con 0x56235e909c00 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdd581b3340 con 0x56235e90bf90 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e9205c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd56d9d640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e97cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5f1e8740 10 client.0 _ll_register_callbacks cb 0x7fdd4c013810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 -- 172.21.15.107:0/818990865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 msgr2=0x56235e97cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e97cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 -- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 msgr2=0x56235e9205c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e9205c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.682+0000 7fdd5659c640 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56235e903e60 con 0x56235e900000 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.683+0000 7fdd5659c640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd48031910 tx=0x7fdd48004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7583c92de80c7215 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.683+0000 7fdd3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.683+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd48035040 con 0x56235e900000 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.683+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdd48003a00 con 0x56235e900000 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.684+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd48004300 con 0x56235e900000 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.684+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd4806b050 con 0x56235e900000 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.684+0000 7fdd5f1e8740 10 client.15990 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:02.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.684+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56235e985a70 con 0x56235e900000 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.684+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdd48004300 con 0x56235e900000 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd4803c440 con 0x56235e900000 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 10 client.15990 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 20 client.15990 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:02.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e9861e0 0x56235e988600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 4 client.15990 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56235e8742d0 con 0x56235e9861e0 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e9861e0 0x56235e988600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd3dffb640 10 client.15990 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.685+0000 7fdd55d9b640 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e9861e0 0x56235e988600 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=669b18a906754d58 server_cookie=34dee0c1d570f79c in_seq=0 out_seq=0 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd3dffb640 1 -- 172.21.15.107:0/818990865 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56235e8742d0 con 0x56235e9861e0 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd3dffb640 10 client.15990 handle_command_reply: tid=0 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 client.15990 shutdown 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 msgr2=0x56235e920040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdd48031910 tx=0x7fdd48004170 comp rx=0 tx=0).stop 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 shutdown_connections 2024-08-20T06:41:02.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x56235e9861e0 0x56235e988600 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56235e90bf90 0x56235e97cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x56235e909c00 0x56235e9205c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 --2- 172.21.15.107:0/818990865 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x56235e900000 0x56235e920040 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd3dffb640 0 client.15990 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 >> 172.21.15.107:0/818990865 conn(0x56235e8fd510 msgr2=0x56235e8fc4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 shutdown_connections 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 1 -- 172.21.15.107:0/818990865 wait complete. 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.686+0000 7fdd5f1e8740 20 client.15990 trim_cache size 0 max 16384 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:41:02.691 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:41:02.692 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:41:02.692 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:41:02.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 msgr2=0x7fdd400b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 0x7fdd400b6110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdd480057c0 tx=0x7fdd480058d0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 msgr2=0x7fdd5810f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000eb10 tx=0x7fdd5000c2d0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 shutdown_connections 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdd58107d60 0x7fdd5810f4f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdd400b3c50 0x7fdd400b6110 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd581073e0 0x7fdd5810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 --2- 172.21.15.107:0/1170725137 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdd581019d0 0x7fdd58110970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 >> 172.21.15.107:0/1170725137 conn(0x7fdd580fd460 msgr2=0x7fdd580ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 shutdown_connections 2024-08-20T06:41:02.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:02.687+0000 7fdd5cdef640 1 -- 172.21.15.107:0/1170725137 wait complete. 2024-08-20T06:41:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:03 smithi183 ceph-mon[22204]: pgmap v453: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 134 op/s 2024-08-20T06:41:03.497 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:03 smithi107 ceph-mon[18495]: pgmap v453: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 134 op/s 2024-08-20T06:41:03.536 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:41:03.537 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:41:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:03 smithi193 ceph-mon[21944]: pgmap v453: 97 pgs: 97 active+clean; 318 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 134 op/s 2024-08-20T06:41:03.902 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 -- 172.21.15.107:0/1555496039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c0103870 msgr2=0x7ff7c0103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 --2- 172.21.15.107:0/1555496039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c0103870 0x7ff7c0103c50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009900 tx=0x7ff7bc02a3f0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 -- 172.21.15.107:0/1555496039 shutdown_connections 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 --2- 172.21.15.107:0/1555496039 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c010ce20 0x7ff7c010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 --2- 172.21.15.107:0/1555496039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0104270 0x7ff7c010c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 --2- 172.21.15.107:0/1555496039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c0103870 0x7ff7c0103c50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.455+0000 7ff7c8133640 1 -- 172.21.15.107:0/1555496039 >> 172.21.15.107:0/1555496039 conn(0x7ff7c00fd460 msgr2=0x7ff7c00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.456+0000 7ff7c8133640 1 -- 172.21.15.107:0/1555496039 shutdown_connections 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.456+0000 7ff7c8133640 1 -- 172.21.15.107:0/1555496039 wait complete. 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.456+0000 7ff7c8133640 1 Processor -- start 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 -- start start 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0103870 0x7ff7c01016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 0x7ff7c0101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c01129a0 con 0x7ff7c010ce20 2024-08-20T06:41:04.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0112820 con 0x7ff7c0104270 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c8133640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0112b20 con 0x7ff7c0103870 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33182/0 (socket says 172.21.15.107:33182) 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 -- 172.21.15.107:0/3669398229 learned_addr learned my addr 172.21.15.107:0/3669398229 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c56ce640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 0x7ff7c0101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c5ecf640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0103870 0x7ff7c01016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 -- 172.21.15.107:0/3669398229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0103870 msgr2=0x7ff7c01016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0103870 0x7ff7c01016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 -- 172.21.15.107:0/3669398229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 msgr2=0x7ff7c0101c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 0x7ff7c0101c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c66d0640 1 -- 172.21.15.107:0/3669398229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01a72b0 con 0x7ff7c010ce20 2024-08-20T06:41:04.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.457+0000 7ff7c56ce640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 0x7ff7c0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7c66d0640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800eb10 tx=0x7ff7b800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004b20 con 0x7ff7c010ce20 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff7b8004cc0 con 0x7ff7c010ce20 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c01a75a0 con 0x7ff7c010ce20 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8010690 con 0x7ff7c010ce20 2024-08-20T06:41:04.463 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.458+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01a7a80 con 0x7ff7c010ce20 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.460+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c0103c50 con 0x7ff7c010ce20 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.462+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff7b8005020 con 0x7ff7c010ce20 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.463+0000 7ff7b6ffd640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 0x7ff7ac0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.463+0000 7ff7c5ecf640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 0x7ff7ac0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.464+0000 7ff7c5ecf640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 0x7ff7ac0b6190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:04.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.464+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff7b80e4160 con 0x7ff7c010ce20 2024-08-20T06:41:04.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.468+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff7b80ab770 con 0x7ff7c010ce20 2024-08-20T06:41:04.849 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.848+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7c002e3a0 con 0x7ff7c010ce20 2024-08-20T06:41:04.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.849+0000 7ff7b6ffd640 1 -- 172.21.15.107:0/3669398229 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11938 (secure 0 0 0) 0x7ff7b80ab070 con 0x7ff7c010ce20 2024-08-20T06:41:04.850 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:41:04.850 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":27,"btime":"2024-08-20T06:39:16:059531+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:39:15.312343+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:41:04.851 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 27 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 msgr2=0x7ff7ac0b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 0x7ff7ac0b6190 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc005e80 comp rx=0 tx=0).stop 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 msgr2=0x7ff7c0102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800eb10 tx=0x7ff7b800c2d0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 shutdown_connections 2024-08-20T06:41:04.853 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff7c010ce20 0x7ff7c0102170 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff7c0104270 0x7ff7c0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff7ac0b3cd0 0x7ff7ac0b6190 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 --2- 172.21.15.107:0/3669398229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7c0103870 0x7ff7c01016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 >> 172.21.15.107:0/3669398229 conn(0x7ff7c00fd460 msgr2=0x7ff7c00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 shutdown_connections 2024-08-20T06:41:04.854 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:04.852+0000 7ff7c8133640 1 -- 172.21.15.107:0/3669398229 wait complete. 2024-08-20T06:41:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:05 smithi183 ceph-mon[22204]: pgmap v454: 97 pgs: 97 active+clean; 326 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-08-20T06:41:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:05 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3669398229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:05 smithi193 ceph-mon[21944]: pgmap v454: 97 pgs: 97 active+clean; 326 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-08-20T06:41:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:05 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3669398229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:05.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:05 smithi107 ceph-mon[18495]: pgmap v454: 97 pgs: 97 active+clean; 326 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.2 MiB/s wr, 153 op/s 2024-08-20T06:41:05.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:05 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3669398229' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:05.837 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:41:05.886 INFO:tasks.workunit.client.0.smithi107.stdout: 60 12966 210 8452 239 4144 456 2024-08-20T06:41:06.201 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 -- 172.21.15.107:0/1695225694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178107d20 msgr2=0x7fa17810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 --2- 172.21.15.107:0/1695225694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178107d20 0x7fa17810e520 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa170009ca0 tx=0x7fa17002a8c0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 -- 172.21.15.107:0/1695225694 shutdown_connections 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 --2- 172.21.15.107:0/1695225694 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178107d20 0x7fa17810e520 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 --2- 172.21.15.107:0/1695225694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 0x7fa1781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 --2- 172.21.15.107:0/1695225694 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178101990 0x7fa178101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.709+0000 7fa17ca09640 1 -- 172.21.15.107:0/1695225694 >> 172.21.15.107:0/1695225694 conn(0x7fa1780fd400 msgr2=0x7fa1780ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.710+0000 7fa17ca09640 1 -- 172.21.15.107:0/1695225694 shutdown_connections 2024-08-20T06:41:06.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.710+0000 7fa17ca09640 1 -- 172.21.15.107:0/1695225694 wait complete. 2024-08-20T06:41:06.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 Processor -- start 2024-08-20T06:41:06.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 -- start start 2024-08-20T06:41:06.713 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 0x7fa178109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 0x7fa17810a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa178110740 con 0x7fa178101990 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781105c0 con 0x7fa178107d20 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17ca09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781108c0 con 0x7fa1781073a0 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa17659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 0x7fa178109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38430/0 (socket says 172.21.15.107:38430) 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.711+0000 7fa176d9d640 1 -- 172.21.15.107:0/2950833602 learned_addr learned my addr 172.21.15.107:0/2950833602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa175d9b640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 0x7fa17810a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 -- 172.21.15.107:0/2950833602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 msgr2=0x7fa17810a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 0x7fa17810a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.714 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 -- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 msgr2=0x7fa178109fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 0x7fa178109fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 -- 172.21.15.107:0/2950833602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1781a2ca0 con 0x7fa178107d20 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa17659c640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 0x7fa178109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa176d9d640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa170032bf0 tx=0x7fa170032c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa1677fe640 1 -- 172.21.15.107:0/2950833602 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170004040 con 0x7fa178107d20 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa1677fe640 1 -- 172.21.15.107:0/2950833602 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa170004670 con 0x7fa178107d20 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa1677fe640 1 -- 172.21.15.107:0/2950833602 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170030bb0 con 0x7fa178107d20 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1781a2f30 con 0x7fa178107d20 2024-08-20T06:41:06.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.712+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781a3420 con 0x7fa178107d20 2024-08-20T06:41:06.716 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.715+0000 7fa17ee02740 1 Processor -- start 2024-08-20T06:41:06.717 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.716+0000 7fa1677fe640 1 -- 172.21.15.107:0/2950833602 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fa1700041e0 con 0x7fa178107d20 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.716+0000 7fa17ee02740 1 -- start start 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb85bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb88520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b49faf0930 con 0x55b49fb7a1a0 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b49fadfe30 con 0x55b49fb79880 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17ee02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b49fae3e70 con 0x55b49fb86130 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38444/0 (socket says 172.21.15.107:38444) 2024-08-20T06:41:06.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 -- 172.21.15.107:0/3798799887 learned_addr learned my addr 172.21.15.107:0/3798799887 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:06.719 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb88520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17659c640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb85bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 -- 172.21.15.107:0/3798799887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 msgr2=0x55b49fb88520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.720 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb88520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 -- 172.21.15.107:0/3798799887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 msgr2=0x55b49fb85bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb85bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 -- 172.21.15.107:0/3798799887 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49f83f650 con 0x55b49fb79880 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa17659c640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb85bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.717+0000 7fa175d9b640 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa16c014850 tx=0x7fa16c009630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=857f2eed2bc92bfc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa1677fe640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 0x7fa1600b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa164ff9640 1 -- 172.21.15.107:0/3798799887 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0140d0 con 0x55b49fb79880 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa164ff9640 1 -- 172.21.15.107:0/3798799887 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa16c014270 con 0x55b49fb79880 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa164ff9640 1 -- 172.21.15.107:0/3798799887 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c0047a0 con 0x55b49fb79880 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa17659c640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 0x7fa1600b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.718+0000 7fa1677fe640 1 -- 172.21.15.107:0/2950833602 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa170105050 con 0x7fa178107d20 2024-08-20T06:41:06.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17659c640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 0x7fa1600b6150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b49fb7ca00 tx=0x7fa168005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- 172.21.15.107:0/3798799887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 msgr2=0x55b49fb79c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa16c014850 tx=0x7fa16c009630 comp rx=0 tx=0).stop 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- 172.21.15.107:0/3798799887 shutdown_connections 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb88520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb85bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 --2- 172.21.15.107:0/3798799887 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb79c60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- 172.21.15.107:0/3798799887 >> 172.21.15.107:0/3798799887 conn(0x55b49fb776b0 msgr2=0x55b49fb78850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- 172.21.15.107:0/3798799887 shutdown_connections 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- 172.21.15.107:0/3798799887 wait complete. 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.719+0000 7fa17ee02740 1 -- start start 2024-08-20T06:41:06.722 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 Processor -- start 2024-08-20T06:41:06.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb835e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb83b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.725 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55b49f942700 con 0x55b49fb7a1a0 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55b49fa66810 con 0x55b49fb79880 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55b49fb88d80 con 0x55b49fb86130 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17ee02740 10 client.0 _ll_register_callbacks cb 0x55b49fad9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17659c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38454/0 (socket says 172.21.15.107:38454) 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa17659c640 1 -- 172.21.15.107:0/898101277 learned_addr learned my addr 172.21.15.107:0/898101277 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.720+0000 7fa175d9b640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb835e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa176d9d640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb83b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 -- 172.21.15.107:0/898101277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 msgr2=0x55b49fb83b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb83b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 -- 172.21.15.107:0/898101277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 msgr2=0x55b49fb835e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.726 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb835e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 -- 172.21.15.107:0/898101277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49fb92160 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17659c640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa16800fa80 tx=0x7fa168004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb56c94e105079e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa15dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16802e060 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa168018030 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.721+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b49fa8f150 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa16801e620 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 10 client.25520 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b49fb94b70 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55b49fa8f150 con 0x55b49fb79880 2024-08-20T06:41:06.727 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa168033050 con 0x55b49fb79880 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 10 client.25520 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 10 client.25520 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 20 client.25520 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fb96030 0x55b49fb98480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 4 client.25520 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b49faf2880 con 0x55b49fb96030 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.722+0000 7fa175d9b640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fb96030 0x55b49fb98480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.723+0000 7fa15dffb640 10 client.25520 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.723+0000 7fa175d9b640 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fb96030 0x55b49fb98480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9e3371470dd8c2fd server_cookie=bf3d811b40db35ce in_seq=0 out_seq=0 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.723+0000 7fa15dffb640 1 -- 172.21.15.107:0/898101277 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b49faf2880 con 0x55b49fb96030 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.723+0000 7fa15dffb640 10 client.25520 handle_command_reply: tid=0 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 client.25520 shutdown 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 msgr2=0x55b49fb830a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa16800fa80 tx=0x7fa168004730 comp rx=0 tx=0).stop 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 shutdown_connections 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb86130 0x55b49fb83b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb7a1a0 0x55b49fb835e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fb96030 0x55b49fb98480 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 --2- 172.21.15.107:0/898101277 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb79880 0x55b49fb830a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa15dffb640 0 client.25520 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 >> 172.21.15.107:0/898101277 conn(0x55b49fb776b0 msgr2=0x55b49fb7d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 shutdown_connections 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 1 -- 172.21.15.107:0/898101277 wait complete. 2024-08-20T06:41:06.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.724+0000 7fa17ee02740 20 client.25520 trim_cache size 0 max 16384 2024-08-20T06:41:06.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.744+0000 7fa17ee02740 1 Processor -- start 2024-08-20T06:41:06.746 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 -- start start 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bf700 con 0x55b49fb86130 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bcfd0 con 0x55b49fb7a1a0 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17ee02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bae00 con 0x55b49fb79880 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32944/0 (socket says 172.21.15.107:32944) 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38462/0 (socket says 172.21.15.107:38462) 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa176d9d640 1 -- 172.21.15.107:0/425409814 learned_addr learned my addr 172.21.15.107:0/425409814 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:06.747 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa17659c640 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 -- 172.21.15.107:0/425409814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 msgr2=0x55b49fb7da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb7da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 -- 172.21.15.107:0/425409814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 msgr2=0x55b49fb9dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa175d9b640 1 -- 172.21.15.107:0/425409814 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b49fb79fd0 con 0x55b49fb7a1a0 2024-08-20T06:41:06.748 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.745+0000 7fa176d9d640 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.746+0000 7fa175d9b640 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00a010 tx=0x7fa16c009a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25101a2f740e898c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.746+0000 7fa15ffff640 1 -- 172.21.15.107:0/425409814 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c02e070 con 0x55b49fb7a1a0 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.746+0000 7fa15ffff640 1 -- 172.21.15.107:0/425409814 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa16c00fbc0 con 0x55b49fb7a1a0 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.746+0000 7fa15ffff640 1 -- 172.21.15.107:0/425409814 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa16c028560 con 0x55b49fb7a1a0 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 -- 172.21.15.107:0/425409814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 msgr2=0x55b49fb84de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00a010 tx=0x7fa16c009a90 comp rx=0 tx=0).stop 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 -- 172.21.15.107:0/425409814 shutdown_connections 2024-08-20T06:41:06.751 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fb9dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb84de0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 --2- 172.21.15.107:0/425409814 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb7da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 -- 172.21.15.107:0/425409814 >> 172.21.15.107:0/425409814 conn(0x55b49fb776b0 msgr2=0x55b49fb846d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 -- 172.21.15.107:0/425409814 shutdown_connections 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.747+0000 7fa17ee02740 1 -- 172.21.15.107:0/425409814 wait complete. 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 -- start start 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 Processor -- start 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb89880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb89e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bb490 con 0x55b49fb86130 2024-08-20T06:41:06.752 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bc900 con 0x55b49fb7a1a0 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781bbb60 con 0x55b49fb79880 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb89e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa17ee02740 10 client.0 _ll_register_callbacks cb 0x7fa168005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa175d9b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb89e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38478/0 (socket says 172.21.15.107:38478) 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.748+0000 7fa176d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32948/0 (socket says 172.21.15.107:32948) 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa175d9b640 1 -- 172.21.15.107:0/1657752703 learned_addr learned my addr 172.21.15.107:0/1657752703 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb89880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 -- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 msgr2=0x55b49fb89e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb89e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 -- 172.21.15.107:0/1657752703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 msgr2=0x55b49fbf8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.753 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 -- 172.21.15.107:0/1657752703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b49fbf9460 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa176d9d640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17659c640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb89880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa168005e00 tx=0x7fa1680296d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154ecb97faf5f972 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa15dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa168039030 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa168024040 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.749+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa16c00fbc0 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa16802e020 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 10 client.25429 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b49fc00860 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa168029d70 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa168033040 con 0x55b49fb79880 2024-08-20T06:41:06.754 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 10 client.25429 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 10 client.25429 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 20 client.25429 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fc01030 0x55b49fc03450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 4 client.25429 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa16c00fbc0 con 0x55b49fc01030 2024-08-20T06:41:06.755 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.750+0000 7fa175d9b640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fc01030 0x55b49fc03450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:06.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa15dffb640 10 client.25429 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:06.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa175d9b640 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fc01030 0x55b49fc03450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cded6b51048edc22 server_cookie=f738acf976e9ffb in_seq=0 out_seq=0 2024-08-20T06:41:06.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa15dffb640 1 -- 172.21.15.107:0/1657752703 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa16c00fbc0 con 0x55b49fc01030 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa15dffb640 10 client.25429 handle_command_reply: tid=0 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 client.25429 shutdown 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 msgr2=0x55b49fb89880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb89880 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa168005e00 tx=0x7fa1680296d0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 shutdown_connections 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55b49fb86130 0x55b49fbf8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55b49fc01030 0x55b49fc03450 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55b49fb7a1a0 0x55b49fb89e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 --2- 172.21.15.107:0/1657752703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b49fb79880 0x55b49fb89880 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa15dffb640 0 client.25429 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 >> 172.21.15.107:0/1657752703 conn(0x55b49fb776b0 msgr2=0x55b49fb75f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 shutdown_connections 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.751+0000 7fa17ee02740 1 -- 172.21.15.107:0/1657752703 wait complete. 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ee02740 20 client.25429 trim_cache size 0 max 16384 2024-08-20T06:41:06.758 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 msgr2=0x7fa1600b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 0x7fa1600b6150 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55b49fb7ca00 tx=0x7fa168005f90 comp rx=0 tx=0).stop 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 msgr2=0x7fa1781a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa170032bf0 tx=0x7fa170032c20 comp rx=0 tx=0).stop 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 shutdown_connections 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa178107d20 0x7fa1781a25a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa1600b3c90 0x7fa1600b6150 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1781073a0 0x7fa17810a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 --2- 172.21.15.107:0/2950833602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa178101990 0x7fa178109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:06.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.752+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 >> 172.21.15.107:0/2950833602 conn(0x7fa1780fd400 msgr2=0x7fa178110ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:06.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.753+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 shutdown_connections 2024-08-20T06:41:06.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:06.753+0000 7fa17ca09640 1 -- 172.21.15.107:0/2950833602 wait complete. 2024-08-20T06:41:07.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:07 smithi183 ceph-mon[22204]: pgmap v455: 97 pgs: 97 active+clean; 329 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 141 op/s 2024-08-20T06:41:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:07 smithi193 ceph-mon[21944]: pgmap v455: 97 pgs: 97 active+clean; 329 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 141 op/s 2024-08-20T06:41:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:07 smithi107 ceph-mon[18495]: pgmap v455: 97 pgs: 97 active+clean; 329 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.8 MiB/s wr, 141 op/s 2024-08-20T06:41:07.679 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:41:08.021 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.601+0000 7f747c36c640 1 -- 172.21.15.107:0/1060885446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 msgr2=0x7f747410f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.601+0000 7f747c36c640 1 --2- 172.21.15.107:0/1060885446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747410f260 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f746c009900 tx=0x7f746c02a410 comp rx=0 tx=0).stop 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 -- 172.21.15.107:0/1060885446 shutdown_connections 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 --2- 172.21.15.107:0/1060885446 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747410f260 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 --2- 172.21.15.107:0/1060885446 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741042c0 0x7f747410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 --2- 172.21.15.107:0/1060885446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741039a0 0x7f7474103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.603 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 -- 172.21.15.107:0/1060885446 >> 172.21.15.107:0/1060885446 conn(0x7f74740fd400 msgr2=0x7f74740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 -- 172.21.15.107:0/1060885446 shutdown_connections 2024-08-20T06:41:08.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.602+0000 7f747c36c640 1 -- 172.21.15.107:0/1060885446 wait complete. 2024-08-20T06:41:08.604 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 Processor -- start 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 -- start start 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741042c0 0x7f747419b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747419d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7474102030 con 0x7f747410ce00 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7474101eb0 con 0x7f74741039a0 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747c36c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74741021b0 con 0x7f74741042c0 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747a108640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747419d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f7479907640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741042c0 0x7f747419b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747a108640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38506/0 (socket says 172.21.15.107:38506) 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.603+0000 7f747a108640 1 -- 172.21.15.107:0/2643987031 learned_addr learned my addr 172.21.15.107:0/2643987031 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:08.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 -- 172.21.15.107:0/2643987031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741042c0 msgr2=0x7f747419b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741042c0 0x7f747419b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 -- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 msgr2=0x7f747419d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747419d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 -- 172.21.15.107:0/2643987031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f747419df20 con 0x7f74741039a0 2024-08-20T06:41:08.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747a108640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f746400e9f0 tx=0x7f746400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f746b7fe640 1 -- 172.21.15.107:0/2643987031 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746400cd80 con 0x7f74741039a0 2024-08-20T06:41:08.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f746b7fe640 1 -- 172.21.15.107:0/2643987031 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74640054b0 con 0x7f74741039a0 2024-08-20T06:41:08.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f746b7fe640 1 -- 172.21.15.107:0/2643987031 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7464010430 con 0x7f74741039a0 2024-08-20T06:41:08.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.604+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f747419e210 con 0x7f74741039a0 2024-08-20T06:41:08.607 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.605+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74741a7b10 con 0x7f74741039a0 2024-08-20T06:41:08.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.607+0000 7f747e765740 1 Processor -- start 2024-08-20T06:41:08.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.607+0000 7f747e765740 1 -- start start 2024-08-20T06:41:08.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x55901008abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x55901008d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55900fff5930 con 0x55901007e880 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55900ffe4e30 con 0x55901007f1a0 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747e765740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55900ffe8e70 con 0x55901008b130 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32972/0 (socket says 172.21.15.107:32972) 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747a909640 1 -- 172.21.15.107:0/4010494598 learned_addr learned my addr 172.21.15.107:0/4010494598 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f7479907640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x55901008d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f746b7fe640 1 -- 172.21.15.107:0/2643987031 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f74640046c0 con 0x7f74741039a0 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.608+0000 7f747a108640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x55901008abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 -- 172.21.15.107:0/4010494598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 msgr2=0x55901008d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x55901008d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 -- 172.21.15.107:0/4010494598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 msgr2=0x55901008abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x55901008abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 -- 172.21.15.107:0/4010494598 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55900fd44650 con 0x55901007e880 2024-08-20T06:41:08.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f7479907640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x55901008d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:08.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f747a909640 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f746c0057c0 tx=0x7f746c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d994bac880960721 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.609+0000 7f7468de6640 1 -- 172.21.15.107:0/4010494598 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c035070 con 0x55901007e880 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.610+0000 7f7468de6640 1 -- 172.21.15.107:0/4010494598 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f746c040030 con 0x55901007e880 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.610+0000 7f7468de6640 1 -- 172.21.15.107:0/4010494598 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c03c860 con 0x55901007e880 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.610+0000 7f746b7fe640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 0x7f74600b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.610+0000 7f746b7fe640 1 -- 172.21.15.107:0/2643987031 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f74640e3f20 con 0x7f74741039a0 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.610+0000 7f747a108640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 0x7f74600b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 -- 172.21.15.107:0/4010494598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 msgr2=0x55901007ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747a108640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 0x7f74600b6190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74640a6eb0 tx=0x7f74640a6cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f746c0057c0 tx=0x7f746c004040 comp rx=0 tx=0).stop 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 -- 172.21.15.107:0/4010494598 shutdown_connections 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x55901008d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x55901008abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 --2- 172.21.15.107:0/4010494598 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x55901007ec60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 -- 172.21.15.107:0/4010494598 >> 172.21.15.107:0/4010494598 conn(0x55901007c6b0 msgr2=0x55901007d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 -- 172.21.15.107:0/4010494598 shutdown_connections 2024-08-20T06:41:08.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.611+0000 7f747e765740 1 -- 172.21.15.107:0/4010494598 wait complete. 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.612+0000 7f747e765740 1 -- start start 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.612+0000 7f747e765740 1 Processor -- start 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.612+0000 7f747e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.612+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.612+0000 7f747e765740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x5590100886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32980/0 (socket says 172.21.15.107:32980) 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 -- 172.21.15.107:0/1323739622 learned_addr learned my addr 172.21.15.107:0/1323739622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x559010088c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55900fe47700 con 0x55901007e880 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55900ff6b810 con 0x55901007f1a0 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55901008ddd0 con 0x55901008b130 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f7479907640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x5590100886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 -- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 msgr2=0x559010088c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x559010088c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 -- 172.21.15.107:0/1323739622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 msgr2=0x5590100886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x5590100886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559010097260 con 0x55901007e880 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 10 client.0 _ll_register_callbacks cb 0x55900ffde700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55900ff94150 con 0x55901007e880 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f7479907640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x5590100886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f747a909640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f746c005c40 tx=0x7f746c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adba1f41f5ce8e78 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.613+0000 7f745dffb640 10 client.16026 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f745dffb640 1 -- 172.21.15.107:0/1323739622 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f746c035070 con 0x55901007e880 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f745dffb640 1 -- 172.21.15.107:0/1323739622 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f746c003ac0 con 0x55901007e880 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f745dffb640 1 -- 172.21.15.107:0/1323739622 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f746c003ce0 con 0x55901007e880 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 10 client.16026 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({fsmap=27}) -- 0x559010099a00 con 0x55901007e880 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f745dffb640 1 -- 172.21.15.107:0/1323739622 <== mon.0 v2:172.21.15.107:3300/0 4 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f746c03c4e0 con 0x55901007e880 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 10 client.16026 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 10 client.16026 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 20 client.16026 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55901009ad50 0x55901009d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 4 client.16026 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.614+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900fff7880 con 0x55901009ad50 2024-08-20T06:41:08.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.615+0000 7f7479907640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55901009ad50 0x55901009d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.615+0000 7f7479907640 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55901009ad50 0x55901009d140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d300fe1e806931a server_cookie=4670c2a6103484d6 in_seq=0 out_seq=0 2024-08-20T06:41:08.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.615+0000 7f745dffb640 10 client.16026 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f745dffb640 1 -- 172.21.15.107:0/1323739622 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55900fff7880 con 0x55901009ad50 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f745dffb640 10 client.16026 handle_command_reply: tid=0 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 client.16026 shutdown 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 msgr2=0x559010088190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f746c005c40 tx=0x7f746c009680 comp rx=0 tx=0).stop 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 shutdown_connections 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55901009ad50 0x55901009d140 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901008b130 0x559010088c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007f1a0 0x5590100886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 --2- 172.21.15.107:0/1323739622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x559010088190 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 >> 172.21.15.107:0/1323739622 conn(0x55901007c6b0 msgr2=0x559010082250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f745dffb640 0 client.16026 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:08.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 shutdown_connections 2024-08-20T06:41:08.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 1 -- 172.21.15.107:0/1323739622 wait complete. 2024-08-20T06:41:08.621 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.616+0000 7f747e765740 20 client.16026 trim_cache size 0 max 16384 2024-08-20T06:41:08.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 Processor -- start 2024-08-20T06:41:08.637 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 -- start start 2024-08-20T06:41:08.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x5590100853a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x5590100858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901008b130 0x5590100897c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747e765740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74741b2a60 con 0x55901007e880 2024-08-20T06:41:08.638 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x5590100853a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747a909640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x5590100853a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:32988/0 (socket says 172.21.15.107:32988) 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.636+0000 7f747a909640 1 -- 172.21.15.107:0/632228298 learned_addr learned my addr 172.21.15.107:0/632228298 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x5590100858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74741b0330 con 0x55901008b130 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74741ae160 con 0x55901007f1a0 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f747a909640 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901008b130 0x5590100897c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 -- 172.21.15.107:0/632228298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901008b130 msgr2=0x5590100897c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901008b130 0x5590100897c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 -- 172.21.15.107:0/632228298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 msgr2=0x5590100853a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x5590100853a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 -- 172.21.15.107:0/632228298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55901008dcc0 con 0x55901007f1a0 2024-08-20T06:41:08.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.637+0000 7f7479907640 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x5590100858e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55901008aa80 tx=0x7f74700105c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b5337c67b9a1727 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.638+0000 7f745ffff640 1 -- 172.21.15.107:0/632228298 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7470004d20 con 0x55901007f1a0 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.638+0000 7f745ffff640 1 -- 172.21.15.107:0/632228298 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74700044d0 con 0x55901007f1a0 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.638+0000 7f745ffff640 1 -- 172.21.15.107:0/632228298 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74700054f0 con 0x55901007f1a0 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 msgr2=0x5590100858e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x5590100858e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55901008aa80 tx=0x7f74700105c0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 shutdown_connections 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901008b130 0x5590100897c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x5590100858e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 --2- 172.21.15.107:0/632228298 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901007e880 0x5590100853a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 >> 172.21.15.107:0/632228298 conn(0x55901007c6b0 msgr2=0x559010083470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 shutdown_connections 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.639+0000 7f747e765740 1 -- 172.21.15.107:0/632228298 wait complete. 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747e765740 1 -- start start 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747e765740 1 Processor -- start 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747e765740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 0x55901009d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747e765740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x55901009dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747e765740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 0x5590100fc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747a909640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 0x55901009d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.640+0000 7f747a909640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 0x55901009d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38546/0 (socket says 172.21.15.107:38546) 2024-08-20T06:41:08.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f74741ae7f0 con 0x55901008b130 2024-08-20T06:41:08.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f74741afc60 con 0x55901007e880 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f74741aeec0 con 0x55901007f1a0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x55901009dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747e765740 10 client.0 _ll_register_callbacks cb 0x7f7470009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747a909640 1 -- 172.21.15.107:0/2170374930 learned_addr learned my addr 172.21.15.107:0/2170374930 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 -- 172.21.15.107:0/2170374930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 msgr2=0x55901009d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 0x55901009d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747a909640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 0x5590100fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 -- 172.21.15.107:0/2170374930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 msgr2=0x5590100fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 0x5590100fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55901007d130 con 0x55901007f1a0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f747a909640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 0x5590100fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f7479907640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x55901009dc80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7470009da0 tx=0x7f7470009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e138ecd13bc49f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.641+0000 7f745dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:08.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f747002c020 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f7470004240 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74700044d0 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7470004a40 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f747e765740 10 client.25444 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5590101050d0 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.642+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f74700044d0 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7470005790 con 0x55901007f1a0 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 10 client.25444 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 10 client.25444 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 20 client.25444 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x559010105790 0x559010107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 4 client.25444 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:08.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55900fff3320 con 0x559010105790 2024-08-20T06:41:08.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f747a909640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x559010105790 0x559010107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:08.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.643+0000 7f745dffb640 10 client.25444 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:08.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747a909640 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x559010105790 0x559010107b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=269a0e3e7dd2c476 server_cookie=93af86b702aef8ee in_seq=0 out_seq=0 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f745dffb640 1 -- 172.21.15.107:0/2170374930 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55900fff3320 con 0x559010105790 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f745dffb640 10 client.25444 handle_command_reply: tid=0 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 client.25444 shutdown 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 msgr2=0x55901009dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x55901009dc80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7470009da0 tx=0x7f7470009f80 comp rx=0 tx=0).stop 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 shutdown_connections 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x559010105790 0x559010107b80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55901008b130 0x5590100fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.648 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55901007f1a0 0x55901009dc80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 --2- 172.21.15.107:0/2170374930 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55901007e880 0x55901009d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 >> 172.21.15.107:0/2170374930 conn(0x55901007c6b0 msgr2=0x55901007af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f745dffb640 0 client.25444 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 shutdown_connections 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 1 -- 172.21.15.107:0/2170374930 wait complete. 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.644+0000 7f747e765740 20 client.25444 trim_cache size 0 max 16384 2024-08-20T06:41:08.649 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:08.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 msgr2=0x7f74600b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.650 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 0x7f74600b6190 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74640a6eb0 tx=0x7f74640a6cf0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 msgr2=0x7f747419b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f746400e9f0 tx=0x7f746400edf0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 shutdown_connections 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.645+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f747410ce00 0x7f747419d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f74600b3cd0 0x7f74600b6190 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74741042c0 0x7f747419b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 --2- 172.21.15.107:0/2643987031 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f74741039a0 0x7f747419b160 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 >> 172.21.15.107:0/2643987031 conn(0x7f74740fd400 msgr2=0x7f74740ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 shutdown_connections 2024-08-20T06:41:08.651 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:08.646+0000 7f747c36c640 1 -- 172.21.15.107:0/2643987031 wait complete. 2024-08-20T06:41:09.473 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:41:09.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:09 smithi183 ceph-mon[22204]: pgmap v456: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 183 op/s 2024-08-20T06:41:09.500 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:09 smithi107 ceph-mon[18495]: pgmap v456: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 183 op/s 2024-08-20T06:41:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:09 smithi193 ceph-mon[21944]: pgmap v456: 97 pgs: 97 active+clean; 340 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 183 op/s 2024-08-20T06:41:09.820 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 -- 172.21.15.107:0/3513154251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781084b0 msgr2=0x7f56781088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 --2- 172.21.15.107:0/3513154251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781084b0 0x7f56781088f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f566c009900 tx=0x7f566c02a3f0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 -- 172.21.15.107:0/3513154251 shutdown_connections 2024-08-20T06:41:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 --2- 172.21.15.107:0/3513154251 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5678108e30 0x7f567810f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 --2- 172.21.15.107:0/3513154251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56781084b0 0x7f56781088f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 --2- 172.21.15.107:0/3513154251 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678102aa0 0x7f5678102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 -- 172.21.15.107:0/3513154251 >> 172.21.15.107:0/3513154251 conn(0x7f56780fe550 msgr2=0x7f5678100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 -- 172.21.15.107:0/3513154251 shutdown_connections 2024-08-20T06:41:10.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.443+0000 7f567dddb640 1 -- 172.21.15.107:0/3513154251 wait complete. 2024-08-20T06:41:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 Processor -- start 2024-08-20T06:41:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 -- start start 2024-08-20T06:41:10.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 0x7f5678111ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678108e30 0x7f56781123e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5678076260 con 0x7f56781084b0 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56780760e0 con 0x7f5678108e30 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f567dddb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56780763e0 con 0x7f5678102aa0 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f56777fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f5677fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678108e30 0x7f56781123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f56777fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:45208/0 (socket says 172.21.15.107:45208) 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.444+0000 7f56777fe640 1 -- 172.21.15.107:0/3816120510 learned_addr learned my addr 172.21.15.107:0/3816120510 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f5676ffd640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 0x7f5678111ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 -- 172.21.15.107:0/3816120510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678108e30 msgr2=0x7f56781123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678108e30 0x7f56781123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 -- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 msgr2=0x7f5678111ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 0x7f5678111ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 -- 172.21.15.107:0/3816120510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56781ab670 con 0x7f5678102aa0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f5676ffd640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 0x7f5678111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f56777fe640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f566800e9f0 tx=0x7f566800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f5674ff9640 1 -- 172.21.15.107:0/3816120510 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566800cd80 con 0x7f5678102aa0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56781ab960 con 0x7f5678102aa0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f5674ff9640 1 -- 172.21.15.107:0/3816120510 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f56680108c0 con 0x7f5678102aa0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.445+0000 7f5674ff9640 1 -- 172.21.15.107:0/3816120510 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680187e0 con 0x7f5678102aa0 2024-08-20T06:41:10.448 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.446+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56781abec0 con 0x7f5678102aa0 2024-08-20T06:41:10.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.448+0000 7f56801d4740 1 Processor -- start 2024-08-20T06:41:10.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 -- start start 2024-08-20T06:41:10.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5674ff9640 1 -- 172.21.15.107:0/3816120510 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f5668010430 con 0x7f5678102aa0 2024-08-20T06:41:10.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c10641a0 0x5562c106fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1072520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562c0fda930 con 0x5562c1063880 2024-08-20T06:41:10.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5562c0fc9e30 con 0x5562c1070130 2024-08-20T06:41:10.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56801d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5562c0fcde70 con 0x5562c10641a0 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33022/0 (socket says 172.21.15.107:33022) 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 -- 172.21.15.107:0/2850161723 learned_addr learned my addr 172.21.15.107:0/2850161723 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f56777fe640 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c10641a0 0x5562c106fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1072520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 -- 172.21.15.107:0/2850161723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c10641a0 msgr2=0x5562c106fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c10641a0 0x5562c106fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 -- 172.21.15.107:0/2850161723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 msgr2=0x5562c1072520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.449+0000 7f5677fff640 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1072520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f5677fff640 1 -- 172.21.15.107:0/2850161723 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c0d29650 con 0x5562c1063880 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f5677fff640 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5670011440 tx=0x7f5670011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31f848a27f256f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f565dffb640 1 -- 172.21.15.107:0/2850161723 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670004d20 con 0x5562c1063880 2024-08-20T06:41:10.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f565dffb640 1 -- 172.21.15.107:0/2850161723 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5670004050 con 0x5562c1063880 2024-08-20T06:41:10.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f565dffb640 1 -- 172.21.15.107:0/2850161723 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567000f780 con 0x5562c1063880 2024-08-20T06:41:10.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.450+0000 7f5674ff9640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 0x7f56600b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56777fe640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 0x7f56600b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 -- 172.21.15.107:0/2850161723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 msgr2=0x5562c1063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f5670011440 tx=0x7f5670011900 comp rx=0 tx=0).stop 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56777fe640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 0x7f56600b6110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5562c1066a00 tx=0x7f56680e9040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 -- 172.21.15.107:0/2850161723 shutdown_connections 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f5674ff9640 1 -- 172.21.15.107:0/3816120510 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f56680e3fe0 con 0x7f5678102aa0 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1072520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c10641a0 0x5562c106fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 --2- 172.21.15.107:0/2850161723 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1063880 0x5562c1063c60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.451+0000 7f56801d4740 1 -- 172.21.15.107:0/2850161723 >> 172.21.15.107:0/2850161723 conn(0x5562c10616b0 msgr2=0x5562c1062850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.452+0000 7f56801d4740 1 -- 172.21.15.107:0/2850161723 shutdown_connections 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.452+0000 7f56801d4740 1 -- 172.21.15.107:0/2850161723 wait complete. 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.452+0000 7f56801d4740 1 -- start start 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.452+0000 7f56801d4740 1 Processor -- start 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1063880 0x5562c106d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1070130 0x5562c106dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5562c0e2c700 con 0x5562c10641a0 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5562c0f50810 con 0x5562c1063880 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5562c1072de0 con 0x5562c1070130 2024-08-20T06:41:10.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f56801d4740 10 client.0 _ll_register_callbacks cb 0x5562c0fc3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33026/0 (socket says 172.21.15.107:33026) 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 -- 172.21.15.107:0/2084183289 learned_addr learned my addr 172.21.15.107:0/2084183289 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5677fff640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1070130 0x5562c106dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5677fff640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1063880 0x5562c106d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 -- 172.21.15.107:0/2084183289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1070130 msgr2=0x5562c106dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1070130 0x5562c106dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 -- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1063880 msgr2=0x5562c106d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1063880 0x5562c106d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.453+0000 7f5676ffd640 1 -- 172.21.15.107:0/2084183289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c107c240 con 0x5562c10641a0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f566c005c40 tx=0x7f566c005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c85be7acf22b186 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f5656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c035040 con 0x5562c10641a0 2024-08-20T06:41:10.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5562c0f79150 con 0x5562c10641a0 2024-08-20T06:41:10.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f566c0048c0 con 0x5562c10641a0 2024-08-20T06:41:10.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f566c0055f0 con 0x5562c10641a0 2024-08-20T06:41:10.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f56801d4740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:10.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.454+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562c107ec40 con 0x5562c10641a0 2024-08-20T06:41:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5562c0f79150 con 0x5562c10641a0 2024-08-20T06:41:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f566c003c50 con 0x5562c10641a0 2024-08-20T06:41:10.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 20 client.16041 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10824d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 4 client.16041 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562c0fdc880 con 0x5562c10800a0 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f5677fff640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.455+0000 7f5656ffd640 10 client.16041 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.456+0000 7f5677fff640 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10824d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bd3b467208f25a server_cookie=83478fe391535477 in_seq=0 out_seq=0 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.456+0000 7f5656ffd640 1 -- 172.21.15.107:0/2084183289 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562c0fdc880 con 0x5562c10800a0 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.456+0000 7f5656ffd640 10 client.16041 handle_command_reply: tid=0 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.456+0000 7f56801d4740 1 client.16041 shutdown 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 msgr2=0x5562c106d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f566c005c40 tx=0x7f566c005270 comp rx=0 tx=0).stop 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 shutdown_connections 2024-08-20T06:41:10.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10824d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1070130 0x5562c106dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f5656ffd640 0 client.16041 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c106d6b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 --2- 172.21.15.107:0/2084183289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1063880 0x5562c106d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 >> 172.21.15.107:0/2084183289 conn(0x5562c10616b0 msgr2=0x5562c1067250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 shutdown_connections 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 1 -- 172.21.15.107:0/2084183289 wait complete. 2024-08-20T06:41:10.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.457+0000 7f56801d4740 20 client.16041 trim_cache size 0 max 16384 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 Processor -- start 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 -- start start 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 0x5562c1087de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1070130 0x5562c1088320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b6eb0 con 0x5562c1070130 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b4780 con 0x5562c10641a0 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f56801d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b25b0 con 0x5562c1063880 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.478 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:45232/0 (socket says 172.21.15.107:45232) 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 -- 172.21.15.107:0/2190163904 learned_addr learned my addr 172.21.15.107:0/2190163904 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 0x5562c1087de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1070130 0x5562c1088320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 -- 172.21.15.107:0/2190163904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 msgr2=0x5562c1087de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 0x5562c1087de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 -- 172.21.15.107:0/2190163904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1070130 msgr2=0x5562c1088320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1070130 0x5562c1088320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.476+0000 7f5677fff640 1 -- 172.21.15.107:0/2190163904 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c10614e0 con 0x5562c1063880 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.477+0000 7f5676ffd640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 0x5562c1087de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.477+0000 7f5677fff640 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5670000f80 tx=0x7f5670005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4242dd7b5ede079 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.477+0000 7f565d7fa640 1 -- 172.21.15.107:0/2190163904 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567001d040 con 0x5562c1063880 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.477+0000 7f565d7fa640 1 -- 172.21.15.107:0/2190163904 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f567000fe90 con 0x5562c1063880 2024-08-20T06:41:10.479 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.477+0000 7f565d7fa640 1 -- 172.21.15.107:0/2190163904 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5670019c60 con 0x5562c1063880 2024-08-20T06:41:10.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 -- 172.21.15.107:0/2190163904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 msgr2=0x5562c1062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.481 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5670000f80 tx=0x7f5670005fd0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 -- 172.21.15.107:0/2190163904 shutdown_connections 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c1070130 0x5562c1088320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c10641a0 0x5562c1087de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 --2- 172.21.15.107:0/2190163904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1062ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 -- 172.21.15.107:0/2190163904 >> 172.21.15.107:0/2190163904 conn(0x5562c10616b0 msgr2=0x5562c10627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 -- 172.21.15.107:0/2190163904 shutdown_connections 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.478+0000 7f56801d4740 1 -- 172.21.15.107:0/2190163904 wait complete. 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 -- start start 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 Processor -- start 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1073030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.482 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b2c40 con 0x5562c10641a0 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b40b0 con 0x5562c1070130 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f56781b3310 con 0x5562c1063880 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.479+0000 7f56801d4740 10 client.0 _ll_register_callbacks cb 0x7f5668005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33042/0 (socket says 172.21.15.107:33042) 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 -- 172.21.15.107:0/1899579716 learned_addr learned my addr 172.21.15.107:0/1899579716 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5677fff640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1073030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5677fff640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 -- 172.21.15.107:0/1899579716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 msgr2=0x5562c1073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.483 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1073030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 -- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 msgr2=0x5562c1073ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1073ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 -- 172.21.15.107:0/1899579716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c1062130 con 0x5562c10641a0 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5676ffd640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f566c0059d0 tx=0x7f566c03cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443dc899d783d577 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5656ffd640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.480+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c035040 con 0x5562c10641a0 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f566c004ad0 con 0x5562c10641a0 2024-08-20T06:41:10.484 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f567000fe90 con 0x5562c10641a0 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f566c009780 con 0x5562c10641a0 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 10 client.16047 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5562c10e9f60 con 0x5562c10641a0 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f566c060070 con 0x5562c10641a0 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f566c045470 con 0x5562c10641a0 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 10 client.16047 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.481+0000 7f56801d4740 20 client.16047 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10ec800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f56801d4740 4 client.16047 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:10.486 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5562c1068d40 con 0x5562c10800a0 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f5677fff640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10ec800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f5656ffd640 10 client.16047 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.482+0000 7f5677fff640 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10ec800 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e161a07f9b5d5cad server_cookie=7dab3c4ff35e2846 in_seq=0 out_seq=0 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f5656ffd640 1 -- 172.21.15.107:0/1899579716 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5562c1068d40 con 0x5562c10800a0 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f5656ffd640 10 client.16047 handle_command_reply: tid=0 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 client.16047 shutdown 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 msgr2=0x5562c1073570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f566c0059d0 tx=0x7f566c03cd30 comp rx=0 tx=0).stop 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 shutdown_connections 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5562c1070130 0x5562c1073ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5562c10641a0 0x5562c1073570 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5562c10800a0 0x5562c10ec800 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 --2- 172.21.15.107:0/1899579716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5562c1063880 0x5562c1073030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 >> 172.21.15.107:0/1899579716 conn(0x5562c10616b0 msgr2=0x5562c1066a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f5656ffd640 0 client.16047 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:10.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 shutdown_connections 2024-08-20T06:41:10.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 1 -- 172.21.15.107:0/1899579716 wait complete. 2024-08-20T06:41:10.488 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.483+0000 7f56801d4740 20 client.16047 trim_cache size 0 max 16384 2024-08-20T06:41:10.488 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 msgr2=0x7f56600b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 0x7f56600b6110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x5562c1066a00 tx=0x7f56680e9040 comp rx=0 tx=0).stop 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 msgr2=0x7f5678111960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f566800e9f0 tx=0x7f566800eeb0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 shutdown_connections 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5678108e30 0x7f56781123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f56781084b0 0x7f5678111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f56600b3c50 0x7f56600b6110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 --2- 172.21.15.107:0/3816120510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5678102aa0 0x7f5678111960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:10.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.484+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 >> 172.21.15.107:0/3816120510 conn(0x7f56780fe550 msgr2=0x7f56781019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:10.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.485+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 shutdown_connections 2024-08-20T06:41:10.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:10.485+0000 7f567dddb640 1 -- 172.21.15.107:0/3816120510 wait complete. 2024-08-20T06:41:11.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:11 smithi107 ceph-mon[18495]: pgmap v457: 97 pgs: 97 active+clean; 342 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-08-20T06:41:11.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:11 smithi183 ceph-mon[22204]: pgmap v457: 97 pgs: 97 active+clean; 342 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-08-20T06:41:11.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:11 smithi193 ceph-mon[21944]: pgmap v457: 97 pgs: 97 active+clean; 342 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 224 op/s 2024-08-20T06:41:12.429 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:41:12.430 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:41:12.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:12 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:41:12.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:12 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:41:12.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:12 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:41:12.803 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:13.313 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:13 smithi107 ceph-mon[18495]: pgmap v458: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 330 op/s 2024-08-20T06:41:13.313 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:41:13.313 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:41:13.314 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 -- 172.21.15.107:0/1788163964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c10ce20 msgr2=0x7f9b0c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 --2- 172.21.15.107:0/1788163964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c10f280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009ca0 tx=0x7f9b0402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 -- 172.21.15.107:0/1788163964 shutdown_connections 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 --2- 172.21.15.107:0/1788163964 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c10f280 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 --2- 172.21.15.107:0/1788163964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 0x7f9b0c10c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 --2- 172.21.15.107:0/1788163964 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.322+0000 7f9b113da640 1 -- 172.21.15.107:0/1788163964 >> 172.21.15.107:0/1788163964 conn(0x7f9b0c0fd400 msgr2=0x7f9b0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.323+0000 7f9b113da640 1 -- 172.21.15.107:0/1788163964 shutdown_connections 2024-08-20T06:41:13.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.323+0000 7f9b113da640 1 -- 172.21.15.107:0/1788163964 wait complete. 2024-08-20T06:41:13.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.323+0000 7f9b113da640 1 Processor -- start 2024-08-20T06:41:13.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 -- start start 2024-08-20T06:41:13.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 0x7f9b0c109030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c109570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c102300 con 0x7f9b0c1039c0 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c102180 con 0x7f9b0c10ce20 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b113da640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c102480 con 0x7f9b0c1042e0 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33062/0 (socket says 172.21.15.107:33062) 2024-08-20T06:41:13.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 -- 172.21.15.107:0/3483510242 learned_addr learned my addr 172.21.15.107:0/3483510242 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 0x7f9b0c109030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c109570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 -- 172.21.15.107:0/3483510242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 msgr2=0x7f9b0c109030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 0x7f9b0c109030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 -- 172.21.15.107:0/3483510242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c10ce20 msgr2=0x7f9b0c109570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c109570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.324+0000 7f9b0affd640 1 -- 172.21.15.107:0/3483510242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b0c1ab6c0 con 0x7f9b0c1039c0 2024-08-20T06:41:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9b0affd640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e9f0 tx=0x7f9afc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9af3fff640 1 -- 172.21.15.107:0/3483510242 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc00cd80 con 0x7f9b0c1039c0 2024-08-20T06:41:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b0c1ab9b0 con 0x7f9b0c1039c0 2024-08-20T06:41:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9af3fff640 1 -- 172.21.15.107:0/3483510242 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9afc0108c0 con 0x7f9b0c1039c0 2024-08-20T06:41:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9af3fff640 1 -- 172.21.15.107:0/3483510242 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9afc0187c0 con 0x7f9b0c1039c0 2024-08-20T06:41:13.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.325+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b0c1abf40 con 0x7f9b0c1039c0 2024-08-20T06:41:13.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.328+0000 7f9af3fff640 1 -- 172.21.15.107:0/3483510242 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f9afc010430 con 0x7f9b0c1039c0 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.328+0000 7f9b137d3740 1 Processor -- start 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.328+0000 7f9b137d3740 1 -- start start 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796823720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e796823c60 0x55e796826050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e79678d940 con 0x55e796823c60 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e79677ce40 con 0x55e7968173b0 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b137d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e796780e80 con 0x55e796817cd0 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b0affd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796823720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38638/0 (socket says 172.21.15.107:38638) 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/2321288344 learned_addr learned my addr 172.21.15.107:0/2321288344 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:13.332 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.329+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e796823c60 0x55e796826050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/2321288344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 msgr2=0x55e796823720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796823720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/2321288344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e796823c60 msgr2=0x55e796826050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e796823c60 0x55e796826050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/2321288344 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e796719520 con 0x55e7968173b0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0affd640 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796823720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9af3fff640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 0x7f9af40b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0affd640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 0x7f9af40b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b04005cc0 tx=0x7f9b04009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53530e07966bfa4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.330+0000 7f9af17fa640 1 -- 172.21.15.107:0/2321288344 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040316d0 con 0x55e7968173b0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9af17fa640 1 -- 172.21.15.107:0/2321288344 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b04036040 con 0x55e7968173b0 2024-08-20T06:41:13.333 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9af17fa640 1 -- 172.21.15.107:0/2321288344 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b040042a0 con 0x55e7968173b0 2024-08-20T06:41:13.334 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9af3fff640 1 -- 172.21.15.107:0/3483510242 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9afc0e4130 con 0x7f9b0c1039c0 2024-08-20T06:41:13.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9b137d3740 1 -- 172.21.15.107:0/2321288344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 msgr2=0x55e796817790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.336 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9b137d3740 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9b04005cc0 tx=0x7f9b04009a20 comp rx=0 tx=0).stop 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.331+0000 7f9b0affd640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 0x7f9af40b6190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0db980 tx=0x7f9afc0e7820 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- 172.21.15.107:0/2321288344 shutdown_connections 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e796823c60 0x55e796826050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796823720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- 172.21.15.107:0/2321288344 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e7968173b0 0x55e796817790 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- 172.21.15.107:0/2321288344 >> 172.21.15.107:0/2321288344 conn(0x55e7968151e0 msgr2=0x55e796816380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- 172.21.15.107:0/2321288344 shutdown_connections 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- 172.21.15.107:0/2321288344 wait complete. 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- start start 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 Processor -- start 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e796820c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796823c60 0x55e7968216d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e7965df890 con 0x55e7968173b0 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e796703700 con 0x55e796817cd0 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.332+0000 7f9b137d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e796826950 con 0x55e796823c60 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b137d3740 10 client.0 _ll_register_callbacks cb 0x55e7967786f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38644/0 (socket says 172.21.15.107:38644) 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 -- 172.21.15.107:0/1640756065 learned_addr learned my addr 172.21.15.107:0/1640756065 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e796820c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 -- 172.21.15.107:0/1640756065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796823c60 msgr2=0x55e7968216d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796823c60 0x55e7968216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 -- 172.21.15.107:0/1640756065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 msgr2=0x55e796820c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e796820c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 -- 172.21.15.107:0/1640756065 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7964dc930 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e796820c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b00012700 tx=0x7f9b00012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c0e0e17ad476de7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00004240 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b04036040 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.333+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9b000043e0 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b000119a0 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 10 client.25574 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e7968325f0 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b04036040 con 0x55e796817cd0 2024-08-20T06:41:13.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9b0000f460 con 0x55e796817cd0 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 10 client.25574 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 20 client.25574 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.334+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968363d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.335+0000 7f9b137d3740 4 client.25574 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.335+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e79678f890 con 0x55e796833f50 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.335+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.335+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968363d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88bc72cbf571908 server_cookie=2676a45d5467c4a2 in_seq=0 out_seq=0 2024-08-20T06:41:13.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.335+0000 7f9aeeff5640 10 client.25574 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:13.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1640756065 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e79678f890 con 0x55e796833f50 2024-08-20T06:41:13.341 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9aeeff5640 10 client.25574 handle_command_reply: tid=0 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 client.25574 shutdown 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 msgr2=0x55e796821190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9b00012700 tx=0x7f9b00012bc0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 shutdown_connections 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968363d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796823c60 0x55e7968216d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796817cd0 0x55e796821190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9aeeff5640 0 client.25574 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:13.342 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1640756065 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e796820c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 >> 172.21.15.107:0/1640756065 conn(0x55e7968151e0 msgr2=0x55e79681ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 shutdown_connections 2024-08-20T06:41:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 1 -- 172.21.15.107:0/1640756065 wait complete. 2024-08-20T06:41:13.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.337+0000 7f9b137d3740 20 client.25574 trim_cache size 0 max 16384 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 Processor -- start 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 -- start start 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e79683bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e79683c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b6f60 con 0x55e7968173b0 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b4830 con 0x55e796823c60 2024-08-20T06:41:13.361 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b137d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b2660 con 0x55e796817cd0 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33094/0 (socket says 172.21.15.107:33094) 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1345547193 learned_addr learned my addr 172.21.15.107:0/1345547193 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e79683bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.360+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e79683c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.362 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1345547193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 msgr2=0x55e79683bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e79683bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1345547193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 msgr2=0x55e79683c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e79683c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1345547193 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79681f150 con 0x55e7968173b0 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e79683bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402ade0 tx=0x7f9b04003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abe852f393b31e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9af0ff9640 1 -- 172.21.15.107:0/1345547193 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04054040 con 0x55e7968173b0 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9af0ff9640 1 -- 172.21.15.107:0/1345547193 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b04036070 con 0x55e7968173b0 2024-08-20T06:41:13.363 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.361+0000 7f9af0ff9640 1 -- 172.21.15.107:0/1345547193 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04052930 con 0x55e7968173b0 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.362+0000 7f9b137d3740 1 -- 172.21.15.107:0/1345547193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 msgr2=0x55e79681fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.362+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9b0402ade0 tx=0x7f9b04003d70 comp rx=0 tx=0).stop 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 -- 172.21.15.107:0/1345547193 shutdown_connections 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e79683c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e79683bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.365 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1345547193 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79681fb40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 -- 172.21.15.107:0/1345547193 >> 172.21.15.107:0/1345547193 conn(0x55e7968151e0 msgr2=0x55e796822aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 -- 172.21.15.107:0/1345547193 shutdown_connections 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 -- 172.21.15.107:0/1345547193 wait complete. 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 -- start start 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.363+0000 7f9b137d3740 1 Processor -- start 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796827850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e796827e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b2cf0 con 0x55e7968173b0 2024-08-20T06:41:13.366 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b4160 con 0x55e796823c60 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f9b0c1b33c0 con 0x55e796817cd0 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b137d3740 10 client.0 _ll_register_callbacks cb 0x7f9b00009790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33098/0 (socket says 172.21.15.107:33098) 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1111024186 learned_addr learned my addr 172.21.15.107:0/1111024186 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796827850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e796827e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1111024186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 msgr2=0x55e796827850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796827850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 msgr2=0x55e796827e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e796827e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.367 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.364+0000 7f9b0b7fe640 1 -- 172.21.15.107:0/1111024186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e796828660 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9b0b7fe640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009620 tx=0x7f9b04009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4bbed18e8d701e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9aeeff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b04054070 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f9b04052440 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b04036070 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9b04031980 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9b137d3740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.365+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e79689edf0 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f9b04036070 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f9b04058020 con 0x55e7968173b0 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:13.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 20 client.16065 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968a0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 4 client.16065 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e79678b330 con 0x55e796833f50 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.366+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968a0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.367+0000 7f9aeeff5640 10 client.16065 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:13.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.367+0000 7f9b0a7fc640 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968a0fd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3593519ddf5a2bed server_cookie=ad42a87f3180709f in_seq=0 out_seq=0 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.374+0000 7f9aeeff5640 1 -- 172.21.15.107:0/1111024186 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e79678b330 con 0x55e796833f50 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.374+0000 7f9aeeff5640 10 client.16065 handle_command_reply: tid=0 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.374+0000 7f9b137d3740 1 client.16065 shutdown 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 msgr2=0x55e79682c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9b04009620 tx=0x7f9b04009920 comp rx=0 tx=0).stop 2024-08-20T06:41:13.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 shutdown_connections 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e796823c60 0x55e796827e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55e796833f50 0x55e7968a0fd0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e796817cd0 0x55e796827850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 --2- 172.21.15.107:0/1111024186 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e7968173b0 0x55e79682c380 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 >> 172.21.15.107:0/1111024186 conn(0x55e7968151e0 msgr2=0x55e79681a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9aeeff5640 0 client.16065 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 shutdown_connections 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 1 -- 172.21.15.107:0/1111024186 wait complete. 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.375+0000 7f9b137d3740 20 client.16065 trim_cache size 0 max 16384 2024-08-20T06:41:13.377 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:41:13.378 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:41:13.378 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "e36a8274-c002-4e87-8b65-1b6a1db10a98", 2024-08-20T06:41:13.378 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:41:13.378 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 msgr2=0x7f9af40b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 0x7f9af40b6190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9afc0db980 tx=0x7f9afc0e7820 comp rx=0 tx=0).stop 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 msgr2=0x7f9b0c108af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00e9f0 tx=0x7f9afc00eeb0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 shutdown_connections 2024-08-20T06:41:13.379 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f9b0c10ce20 0x7f9b0c109570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f9af40b3cd0 0x7f9af40b6190 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b0c1042e0 0x7f9b0c109030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 --2- 172.21.15.107:0/3483510242 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9b0c1039c0 0x7f9b0c108af0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:13.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 >> 172.21.15.107:0/3483510242 conn(0x7f9b0c0fd400 msgr2=0x7f9b0c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:13.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 shutdown_connections 2024-08-20T06:41:13.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:13.376+0000 7f9b113da640 1 -- 172.21.15.107:0/3483510242 wait complete. 2024-08-20T06:41:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:13 smithi193 ceph-mon[21944]: pgmap v458: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 330 op/s 2024-08-20T06:41:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:41:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:41:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:41:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:13 smithi183 ceph-mon[22204]: pgmap v458: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.0 MiB/s wr, 330 op/s 2024-08-20T06:41:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:41:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:41:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:41:14.271 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:41:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:14 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:14.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:14 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.634 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:14 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:15.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.203+0000 7fdc2d068640 1 -- 172.21.15.107:0/2271096656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc2811bdd0 msgr2=0x7fdc2811e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.203+0000 7fdc2d068640 1 --2- 172.21.15.107:0/2271096656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc2811bdd0 0x7fdc2811e1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009ca0 tx=0x7fdc2002a940 comp rx=0 tx=0).stop 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.204+0000 7fdc2d068640 1 -- 172.21.15.107:0/2271096656 shutdown_connections 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.204+0000 7fdc2d068640 1 --2- 172.21.15.107:0/2271096656 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc2811bdd0 0x7fdc2811e1c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.204+0000 7fdc2d068640 1 --2- 172.21.15.107:0/2271096656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc2810a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.204+0000 7fdc2d068640 1 --2- 172.21.15.107:0/2271096656 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc28109e80 0x7fdc28110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.204+0000 7fdc2d068640 1 -- 172.21.15.107:0/2271096656 >> 172.21.15.107:0/2271096656 conn(0x7fdc2806dcc0 msgr2=0x7fdc2806e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.205+0000 7fdc2d068640 1 -- 172.21.15.107:0/2271096656 shutdown_connections 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.205+0000 7fdc2d068640 1 -- 172.21.15.107:0/2271096656 wait complete. 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.206+0000 7fdc2d068640 1 Processor -- start 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.206+0000 7fdc2d068640 1 -- start start 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.206+0000 7fdc2d068640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc28109e80 0x7fdc28119230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.206+0000 7fdc2d068640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.206+0000 7fdc2d068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc2811bdd0 0x7fdc28112600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:45320/0 (socket says 172.21.15.107:45320) 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 -- 172.21.15.107:0/4207587504 learned_addr learned my addr 172.21.15.107:0/4207587504 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:15.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc28109e80 0x7fdc28119230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2759e640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc2811bdd0 0x7fdc28112600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc28121760 con 0x7fdc2811bdd0 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281215e0 con 0x7fdc28109e80 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281218e0 con 0x7fdc2810a260 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 -- 172.21.15.107:0/4207587504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc28109e80 msgr2=0x7fdc28119230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc28109e80 0x7fdc28119230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 -- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc2811bdd0 msgr2=0x7fdc28112600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc2811bdd0 0x7fdc28112600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.207+0000 7fdc2659c640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc28112ec0 con 0x7fdc2810a260 2024-08-20T06:41:15.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.208+0000 7fdc2659c640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ea60 tx=0x7fdc1800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.210+0000 7fdc0ffff640 1 -- 172.21.15.107:0/4207587504 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1800cde0 con 0x7fdc2810a260 2024-08-20T06:41:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.210+0000 7fdc0ffff640 1 -- 172.21.15.107:0/4207587504 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc180035f0 con 0x7fdc2810a260 2024-08-20T06:41:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.210+0000 7fdc0ffff640 1 -- 172.21.15.107:0/4207587504 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18010690 con 0x7fdc2810a260 2024-08-20T06:41:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.210+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc281131b0 con 0x7fdc2810a260 2024-08-20T06:41:15.213 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.210+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc281180d0 con 0x7fdc2810a260 2024-08-20T06:41:15.216 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.215+0000 7fdc0ffff640 1 -- 172.21.15.107:0/4207587504 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fdc18004000 con 0x7fdc2810a260 2024-08-20T06:41:15.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.217+0000 7fdc0ffff640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 0x7fdc100b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.217+0000 7fdc0ffff640 1 -- 172.21.15.107:0/4207587504 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdc180e3dd0 con 0x7fdc2810a260 2024-08-20T06:41:15.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 Processor -- start 2024-08-20T06:41:15.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 -- start start 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa2123d8e0 con 0x55fa212c7150 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fa2122cde0 con 0x55fa212d30e0 2024-08-20T06:41:15.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.218+0000 7fdc2f461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa21230e20 con 0x55fa212c6830 2024-08-20T06:41:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.220+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.220+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33126/0 (socket says 172.21.15.107:33126) 2024-08-20T06:41:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.220+0000 7fdc2659c640 1 -- 172.21.15.107:0/3416510981 learned_addr learned my addr 172.21.15.107:0/3416510981 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:15.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 0x7fdc100b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc2759e640 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3416510981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 msgr2=0x55fa212d54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3416510981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 msgr2=0x55fa212d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3416510981 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa20f8c650 con 0x55fa212c6830 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 0x7fdc100b6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c014580 tx=0x7fdc1c014980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.221+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212c6c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c010840 tx=0x7fdc1c01a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a4be2cdcf3c8fc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.222+0000 7fdc0d7fa640 1 -- 172.21.15.107:0/3416510981 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c010e80 con 0x55fa212c6830 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.222+0000 7fdc0d7fa640 1 -- 172.21.15.107:0/3416510981 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc1c014170 con 0x55fa212c6830 2024-08-20T06:41:15.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.222+0000 7fdc0d7fa640 1 -- 172.21.15.107:0/3416510981 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c004b80 con 0x55fa212c6830 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 -- 172.21.15.107:0/3416510981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 msgr2=0x55fa212c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212c6c10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c010840 tx=0x7fdc1c01a000 comp rx=0 tx=0).stop 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 -- 172.21.15.107:0/3416510981 shutdown_connections 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3416510981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212c6c10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 -- 172.21.15.107:0/3416510981 >> 172.21.15.107:0/3416510981 conn(0x55fa212c4660 msgr2=0x55fa212c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 -- 172.21.15.107:0/3416510981 shutdown_connections 2024-08-20T06:41:15.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.223+0000 7fdc2f461740 1 -- 172.21.15.107:0/3416510981 wait complete. 2024-08-20T06:41:15.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 -- start start 2024-08-20T06:41:15.225 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 Processor -- start 2024-08-20T06:41:15.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212d0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55fa2108f700 con 0x55fa212d30e0 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55fa211b37c0 con 0x55fa212c6830 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55fa212d5d30 con 0x55fa212c7150 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2f461740 10 client.0 _ll_register_callbacks cb 0x55fa212266b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38708/0 (socket says 172.21.15.107:38708) 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc26d9d640 1 -- 172.21.15.107:0/6226702 learned_addr learned my addr 172.21.15.107:0/6226702 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.224+0000 7fdc2759e640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 -- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 msgr2=0x55fa212d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212d0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 -- 172.21.15.107:0/6226702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 msgr2=0x55fa212d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 -- 172.21.15.107:0/6226702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa212df0a0 con 0x55fa212c7150 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc2659c640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212d0570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc180aa9e0 tx=0x7fdc180043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9cb87055b15149 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.225+0000 7fdc0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc180c5020 con 0x55fa212c7150 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdc180037a0 con 0x55fa212c7150 2024-08-20T06:41:15.228 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fa211dc100 con 0x55fa212c7150 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc180aace0 con 0x55fa212c7150 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc2f461740 10 client.25489 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa212e1a50 con 0x55fa212c7150 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55fa211dc100 con 0x55fa212c7150 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.226+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc180a99b0 con 0x55fa212c7150 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 10 client.25489 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 10 client.25489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 20 client.25489 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:15.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa212e2f00 0x55fa212e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 4 client.25489 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:15.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa2123f830 con 0x55fa212e2f00 2024-08-20T06:41:15.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.227+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa212e2f00 0x55fa212e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.228+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa212e2f00 0x55fa212e52f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a8b851751703ea5 server_cookie=bf4878ace0b63a68 in_seq=0 out_seq=0 2024-08-20T06:41:15.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.228+0000 7fdc0aff5640 10 client.25489 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.230+0000 7fdc0aff5640 1 -- 172.21.15.107:0/6226702 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa2123f830 con 0x55fa212e2f00 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.230+0000 7fdc0aff5640 10 client.25489 handle_command_reply: tid=0 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.230+0000 7fdc2f461740 1 client.25489 shutdown 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 msgr2=0x55fa212d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212d0570 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc180aa9e0 tx=0x7fdc180043f0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 shutdown_connections 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa212e2f00 0x55fa212e52f0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212d0570 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 --2- 172.21.15.107:0/6226702 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 >> 172.21.15.107:0/6226702 conn(0x55fa212c4660 msgr2=0x55fa212ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc0aff5640 0 client.25489 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:15.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 shutdown_connections 2024-08-20T06:41:15.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 1 -- 172.21.15.107:0/6226702 wait complete. 2024-08-20T06:41:15.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.231+0000 7fdc2f461740 20 client.25489 trim_cache size 0 max 16384 2024-08-20T06:41:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.258+0000 7fdc2f461740 1 Processor -- start 2024-08-20T06:41:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.258+0000 7fdc2f461740 1 -- start start 2024-08-20T06:41:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.258+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.258+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.258+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212cd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2f461740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b6fd0 con 0x55fa212d30e0 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2f461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b48a0 con 0x55fa212c6830 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2f461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b26d0 con 0x55fa212c7150 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:38714/0 (socket says 172.21.15.107:38714) 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3039370407 learned_addr learned my addr 172.21.15.107:0/3039370407 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:15.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2759e640 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3039370407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 msgr2=0x55fa212eb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212eb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3039370407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 msgr2=0x55fa212cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212cd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3039370407 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa212ce530 con 0x55fa212c6830 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc2659c640 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.259+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b910 tx=0x7fdc1c016040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65bb7fd9daa0f1f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc0cff9640 1 -- 172.21.15.107:0/3039370407 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00bc00 con 0x55fa212c6830 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc0cff9640 1 -- 172.21.15.107:0/3039370407 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc1c014170 con 0x55fa212c6830 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc0cff9640 1 -- 172.21.15.107:0/3039370407 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c020470 con 0x55fa212c6830 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc2f461740 1 -- 172.21.15.107:0/3039370407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 msgr2=0x55fa212eabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00b910 tx=0x7fdc1c016040 comp rx=0 tx=0).stop 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.260+0000 7fdc2f461740 1 -- 172.21.15.107:0/3039370407 shutdown_connections 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212d30e0 0x55fa212cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c7150 0x55fa212eb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3039370407 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212c6830 0x55fa212eabe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 -- 172.21.15.107:0/3039370407 >> 172.21.15.107:0/3039370407 conn(0x55fa212c4660 msgr2=0x55fa212cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 -- 172.21.15.107:0/3039370407 shutdown_connections 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 -- 172.21.15.107:0/3039370407 wait complete. 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 -- start start 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.261+0000 7fdc2f461740 1 Processor -- start 2024-08-20T06:41:15.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b2d60 con 0x55fa212c7150 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b41d0 con 0x55fa212d30e0 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fdc281b3430 con 0x55fa212c6830 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2f461740 10 client.0 _ll_register_callbacks cb 0x7fdc180aa450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2759e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:45360/0 (socket says 172.21.15.107:45360) 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3460430843 learned_addr learned my addr 172.21.15.107:0/3460430843 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc2659c640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 msgr2=0x55fa212d6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3460430843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 msgr2=0x55fa212d5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.262+0000 7fdc26d9d640 1 -- 172.21.15.107:0/3460430843 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa2134bba0 con 0x55fa212c6830 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc2759e640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc26d9d640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005840 tx=0x7fdc1c00ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4acbb281db3b72e6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc0aff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c03f060 con 0x55fa212c6830 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fdc1c025040 con 0x55fa212c6830 2024-08-20T06:41:15.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc1c014170 con 0x55fa212c6830 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdc1c00ee30 con 0x55fa212c6830 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc2f461740 10 client.25495 fetch_fsmap learned FSMap version 27 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.263+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa2134ce60 con 0x55fa212c6830 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fdc1c014170 con 0x55fa212c6830 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdc1c020970 con 0x55fa212c6830 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 10 client.25495 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 10 client.25495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:15.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 20 client.25495 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa2134d2f0 0x55fa2134f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 4 client.25495 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa212bf830 con 0x55fa2134d2f0 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.264+0000 7fdc2659c640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa2134d2f0 0x55fa2134f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.265+0000 7fdc2659c640 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa2134d2f0 0x55fa2134f710 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b4cde4359f579ba server_cookie=c9510786a84bae4 in_seq=0 out_seq=0 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.265+0000 7fdc0aff5640 10 client.25495 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc0aff5640 1 -- 172.21.15.107:0/3460430843 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55fa212bf830 con 0x55fa2134d2f0 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc0aff5640 10 client.25495 handle_command_reply: tid=0 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 client.25495 shutdown 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 msgr2=0x55fa212db8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005840 tx=0x7fdc1c00ba80 comp rx=0 tx=0).stop 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 shutdown_connections 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55fa212d30e0 0x55fa212d6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55fa212c7150 0x55fa212d5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55fa2134d2f0 0x55fa2134f710 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc2f461740 1 --2- 172.21.15.107:0/3460430843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fa212c6830 0x55fa212db8b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.266+0000 7fdc0aff5640 0 client.25495 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.267+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 >> 172.21.15.107:0/3460430843 conn(0x55fa212c4660 msgr2=0x55fa212c2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.267+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 shutdown_connections 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.267+0000 7fdc2f461740 1 -- 172.21.15.107:0/3460430843 wait complete. 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.267+0000 7fdc2f461740 20 client.25495 trim_cache size 0 max 16384 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:41:15.270 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (2838 inodes in the stack)", 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "e36a8274-c002-4e87-8b65-1b6a1db10a98": { 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "e36a8274-c002-4e87-8b65-1b6a1db10a98", 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:41:15.271 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 msgr2=0x7fdc100b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 0x7fdc100b6260 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c014580 tx=0x7fdc1c014980 comp rx=0 tx=0).stop 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 msgr2=0x7fdc28119770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800ea60 tx=0x7fdc1800ef20 comp rx=0 tx=0).stop 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 shutdown_connections 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdc2811bdd0 0x7fdc28112600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fdc100b3da0 0x7fdc100b6260 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc2810a260 0x7fdc28119770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 --2- 172.21.15.107:0/4207587504 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fdc28109e80 0x7fdc28119230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:15.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 >> 172.21.15.107:0/4207587504 conn(0x7fdc2806dcc0 msgr2=0x7fdc2811c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:15.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 shutdown_connections 2024-08-20T06:41:15.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:15.268+0000 7fdc2d068640 1 -- 172.21.15.107:0/4207587504 wait complete. 2024-08-20T06:41:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:15 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:41:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:15 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:15 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:15 smithi107 ceph-mon[18495]: pgmap v459: 97 pgs: 97 active+clean; 347 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-08-20T06:41:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:15 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:41:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:15 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:15 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:15 smithi193 ceph-mon[21944]: pgmap v459: 97 pgs: 97 active+clean; 347 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-08-20T06:41:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:15 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:41:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:15 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:15 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:15 smithi183 ceph-mon[22204]: pgmap v459: 97 pgs: 97 active+clean; 347 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 513 op/s 2024-08-20T06:41:15.886 INFO:tasks.workunit.client.0.smithi107.stdout: 66 11334 229 7784 147 4370 311 2024-08-20T06:41:16.647 INFO:tasks.cephfs.filesystem:scrub status for tag:e36a8274-c002-4e87-8b65-1b6a1db10a98 - {'path': '/', 'tag': 'e36a8274-c002-4e87-8b65-1b6a1db10a98', 'options': 'recursive,force'} 2024-08-20T06:41:17.538 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:17 smithi183 ceph-mon[22204]: pgmap v460: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 607 op/s 2024-08-20T06:41:17.538 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:17 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:17 smithi193 ceph-mon[21944]: pgmap v460: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 607 op/s 2024-08-20T06:41:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:17 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:17 smithi107 ceph-mon[18495]: pgmap v460: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 607 op/s 2024-08-20T06:41:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:17 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:19 smithi193 ceph-mon[21944]: pgmap v461: 97 pgs: 97 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 814 op/s 2024-08-20T06:41:19.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:19 smithi107 ceph-mon[18495]: pgmap v461: 97 pgs: 97 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 814 op/s 2024-08-20T06:41:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:19 smithi183 ceph-mon[22204]: pgmap v461: 97 pgs: 97 active+clean; 345 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.8 MiB/s wr, 814 op/s 2024-08-20T06:41:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:21 smithi193 ceph-mon[21944]: pgmap v462: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 939 op/s 2024-08-20T06:41:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:21 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:41:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:21 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:41:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:21 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:41:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:21 smithi107 ceph-mon[18495]: pgmap v462: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 939 op/s 2024-08-20T06:41:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:21 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:41:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:21 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:41:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:21 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:41:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:21 smithi183 ceph-mon[22204]: pgmap v462: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 939 op/s 2024-08-20T06:41:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:21 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:41:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:21 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:41:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:21 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:41:22.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:22 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:22.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:22 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:22.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:22 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:23 smithi193 ceph-mon[21944]: pgmap v463: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 971 op/s 2024-08-20T06:41:23.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:23 smithi107 ceph-mon[18495]: pgmap v463: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 971 op/s 2024-08-20T06:41:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:23 smithi183 ceph-mon[22204]: pgmap v463: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 971 op/s 2024-08-20T06:41:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:25 smithi193 ceph-mon[21944]: pgmap v464: 97 pgs: 97 active+clean; 338 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 963 op/s 2024-08-20T06:41:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:25 smithi107 ceph-mon[18495]: pgmap v464: 97 pgs: 97 active+clean; 338 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 963 op/s 2024-08-20T06:41:25.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:25 smithi183 ceph-mon[22204]: pgmap v464: 97 pgs: 97 active+clean; 338 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.0 MiB/s wr, 963 op/s 2024-08-20T06:41:25.886 INFO:tasks.workunit.client.0.smithi107.stdout: 70 13186 295 8226 111 4180 361 2024-08-20T06:41:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:27 smithi193 ceph-mon[21944]: pgmap v465: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-08-20T06:41:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:27 smithi107 ceph-mon[18495]: pgmap v465: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-08-20T06:41:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:27 smithi183 ceph-mon[22204]: pgmap v465: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 813 op/s 2024-08-20T06:41:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:29 smithi193 ceph-mon[21944]: pgmap v466: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 771 op/s 2024-08-20T06:41:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:29 smithi107 ceph-mon[18495]: pgmap v466: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 771 op/s 2024-08-20T06:41:29.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:29 smithi183 ceph-mon[22204]: pgmap v466: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.6 MiB/s wr, 771 op/s 2024-08-20T06:41:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:31 smithi193 ceph-mon[21944]: pgmap v467: 97 pgs: 97 active+clean; 354 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 605 op/s 2024-08-20T06:41:31.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:31 smithi107 ceph-mon[18495]: pgmap v467: 97 pgs: 97 active+clean; 354 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 605 op/s 2024-08-20T06:41:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:31 smithi183 ceph-mon[22204]: pgmap v467: 97 pgs: 97 active+clean; 354 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 605 op/s 2024-08-20T06:41:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:33 smithi193 ceph-mon[21944]: pgmap v468: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 493 op/s 2024-08-20T06:41:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:33 smithi107 ceph-mon[18495]: pgmap v468: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 493 op/s 2024-08-20T06:41:33.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:33 smithi183 ceph-mon[22204]: pgmap v468: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 493 op/s 2024-08-20T06:41:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:35 smithi193 ceph-mon[21944]: pgmap v469: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-08-20T06:41:35.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:35 smithi107 ceph-mon[18495]: pgmap v469: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-08-20T06:41:35.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:35 smithi183 ceph-mon[22204]: pgmap v469: 97 pgs: 97 active+clean; 346 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.4 MiB/s wr, 501 op/s 2024-08-20T06:41:35.886 INFO:tasks.workunit.client.0.smithi107.stdout: 73 13175 217 8015 130 4394 393 2024-08-20T06:41:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:37 smithi193 ceph-mon[21944]: pgmap v470: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-08-20T06:41:37.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:37 smithi107 ceph-mon[18495]: pgmap v470: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-08-20T06:41:37.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:37 smithi183 ceph-mon[22204]: pgmap v470: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.7 MiB/s wr, 441 op/s 2024-08-20T06:41:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:39 smithi193 ceph-mon[21944]: pgmap v471: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-08-20T06:41:39.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:39 smithi107 ceph-mon[18495]: pgmap v471: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-08-20T06:41:39.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:39 smithi183 ceph-mon[22204]: pgmap v471: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.9 MiB/s wr, 452 op/s 2024-08-20T06:41:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:41 smithi193 ceph-mon[21944]: pgmap v472: 97 pgs: 97 active+clean; 353 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 401 op/s 2024-08-20T06:41:41.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:41 smithi107 ceph-mon[18495]: pgmap v472: 97 pgs: 97 active+clean; 353 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 401 op/s 2024-08-20T06:41:41.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:41 smithi183 ceph-mon[22204]: pgmap v472: 97 pgs: 97 active+clean; 353 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.2 MiB/s wr, 401 op/s 2024-08-20T06:41:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:43 smithi193 ceph-mon[21944]: pgmap v473: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-08-20T06:41:43.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:43 smithi107 ceph-mon[18495]: pgmap v473: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-08-20T06:41:43.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:43 smithi183 ceph-mon[22204]: pgmap v473: 97 pgs: 97 active+clean; 344 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 391 op/s 2024-08-20T06:41:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:45 smithi193 ceph-mon[21944]: pgmap v474: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-08-20T06:41:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:45 smithi107 ceph-mon[18495]: pgmap v474: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-08-20T06:41:45.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:45 smithi183 ceph-mon[22204]: pgmap v474: 97 pgs: 97 active+clean; 348 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 421 op/s 2024-08-20T06:41:45.886 INFO:tasks.workunit.client.0.smithi107.stdout: 76 12560 186 7884 159 5939 381 2024-08-20T06:41:46.647 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:41:46.995 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:47.395 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:47 smithi107 ceph-mon[18495]: pgmap v475: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-08-20T06:41:47.395 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:47 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 -- 172.21.15.107:0/533434864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c01073e0 msgr2=0x7f54c0107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 --2- 172.21.15.107:0/533434864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c01073e0 0x7f54c0107820 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009900 tx=0x7f54b402a410 comp rx=0 tx=0).stop 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 -- 172.21.15.107:0/533434864 shutdown_connections 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 --2- 172.21.15.107:0/533434864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c0107d60 0x7f54c010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 --2- 172.21.15.107:0/533434864 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c01073e0 0x7f54c0107820 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 --2- 172.21.15.107:0/533434864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 -- 172.21.15.107:0/533434864 >> 172.21.15.107:0/533434864 conn(0x7f54c00fd460 msgr2=0x7f54c00ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 -- 172.21.15.107:0/533434864 shutdown_connections 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 -- 172.21.15.107:0/533434864 wait complete. 2024-08-20T06:41:47.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.504+0000 7f54c6697640 1 Processor -- start 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 -- start start 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0116b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 0x7f54c010d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c0107d60 0x7f54c010dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54c006bad0 con 0x7f54c01073e0 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54c006b950 con 0x7f54c0107d60 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54c6697640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54c006bc50 con 0x7f54c01019d0 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54bffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0116b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54bffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0116b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:38018/0 (socket says 172.21.15.107:38018) 2024-08-20T06:41:47.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54bffff640 1 -- 172.21.15.107:0/1560641895 learned_addr learned my addr 172.21.15.107:0/1560641895 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.505+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 0x7f54c010d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c0107d60 0x7f54c010dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1560641895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 msgr2=0x7f54c0116b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0116b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1560641895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c0107d60 msgr2=0x7f54c010dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c0107d60 0x7f54c010dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1560641895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c010e540 con 0x7f54c01073e0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 0x7f54c010d830 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a930 tx=0x7f54b4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54bd7fa640 1 -- 172.21.15.107:0/1560641895 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b403f070 con 0x7f54c01073e0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.506+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01ab9a0 con 0x7f54c01073e0 2024-08-20T06:41:47.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.507+0000 7f54bd7fa640 1 -- 172.21.15.107:0/1560641895 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54b403baf0 con 0x7f54c01073e0 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.507+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01abe80 con 0x7f54c01073e0 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.507+0000 7f54bd7fa640 1 -- 172.21.15.107:0/1560641895 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b4049060 con 0x7f54c01073e0 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.509+0000 7f54bd7fa640 1 -- 172.21.15.107:0/1560641895 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f54b4042070 con 0x7f54c01073e0 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 Processor -- start 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 -- start start 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 0x559a507ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507f90e0 0x559a507fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a507638e0 con 0x559a507ed150 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a50752de0 con 0x559a507f90e0 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.510+0000 7f54c8a90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a50756e20 con 0x559a507ec830 2024-08-20T06:41:47.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44736/0 (socket says 172.21.15.107:44736) 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bffff640 1 -- 172.21.15.107:0/1452341272 learned_addr learned my addr 172.21.15.107:0/1452341272 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507f90e0 0x559a507fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 0x559a507ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 -- 172.21.15.107:0/1452341272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507f90e0 msgr2=0x559a507fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507f90e0 0x559a507fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 -- 172.21.15.107:0/1452341272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 msgr2=0x559a507f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 -- 172.21.15.107:0/1452341272 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a504b2650 con 0x559a507ec830 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bffff640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bd7fa640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 0x7f54a80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 0x7f54a80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 0x559a507ecc10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009f40 tx=0x7f54b80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ccb740f6f34e0c3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.511+0000 7f54bd7fa640 1 -- 172.21.15.107:0/1560641895 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f54b4103d20 con 0x7f54c01073e0 2024-08-20T06:41:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54a67fc640 1 -- 172.21.15.107:0/1452341272 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8004250 con 0x559a507ec830 2024-08-20T06:41:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54a67fc640 1 -- 172.21.15.107:0/1452341272 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54b8003e30 con 0x559a507ec830 2024-08-20T06:41:47.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54a67fc640 1 -- 172.21.15.107:0/1452341272 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b801b430 con 0x559a507ec830 2024-08-20T06:41:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54c8a90740 1 -- 172.21.15.107:0/1452341272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 msgr2=0x559a507ecc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 0x559a507ecc10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009f40 tx=0x7f54b80118d0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.512+0000 7f54c4c34640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 0x7f54a80b6090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f54b8020000 tx=0x7f54b800ffd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 -- 172.21.15.107:0/1452341272 shutdown_connections 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507f90e0 0x559a507fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1452341272 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ec830 0x559a507ecc10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 -- 172.21.15.107:0/1452341272 >> 172.21.15.107:0/1452341272 conn(0x559a507ea660 msgr2=0x559a507eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 -- 172.21.15.107:0/1452341272 shutdown_connections 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 -- 172.21.15.107:0/1452341272 wait complete. 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 -- start start 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.513+0000 7f54c8a90740 1 Processor -- start 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 0x559a507f6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ed150 0x559a507f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.523 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a507f6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x559a505b5700 con 0x559a507ec830 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 0x559a507f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 0x559a507f6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44750/0 (socket says 172.21.15.107:44750) 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ed150 0x559a507f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54bffff640 1 -- 172.21.15.107:0/3220985870 learned_addr learned my addr 172.21.15.107:0/3220985870 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x559a506d97c0 con 0x559a507ed150 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x559a507fbd90 con 0x559a507f90e0 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a507f6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c8a90740 10 client.0 _ll_register_callbacks cb 0x559a5074c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 -- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ed150 msgr2=0x559a507f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ed150 0x559a507f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 -- 172.21.15.107:0/3220985870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 msgr2=0x559a507f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 0x559a507f6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.514+0000 7f54c4c34640 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a508051d0 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f54c4c34640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a507f6ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009f10 tx=0x7f54b8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6840e2f557130a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f549f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8036020 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f54b8019840 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559a50702100 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54b8019a40 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f54c8a90740 10 client.25507 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a50807b40 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.515+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54b802c040 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f54b8019c50 con 0x559a507f90e0 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 10 client.25507 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:47.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 20 client.25507 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a50808ef0 0x559a5080b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 4 client.25507 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a50702100 con 0x559a50808ef0 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54bffff640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a50808ef0 0x559a5080b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f549f7fe640 10 client.25507 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.516+0000 7f54bffff640 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a50808ef0 0x559a5080b2e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff00afc9102ac474 server_cookie=8455f0483acea019 in_seq=0 out_seq=0 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f549f7fe640 1 -- 172.21.15.107:0/3220985870 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559a50702100 con 0x559a50808ef0 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f549f7fe640 10 client.25507 handle_command_reply: tid=0 2024-08-20T06:41:47.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f54c8a90740 1 client.25507 shutdown 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 msgr2=0x559a507f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a507f6ae0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009f10 tx=0x7f54b8017000 comp rx=0 tx=0).stop 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.517+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 shutdown_connections 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a50808ef0 0x559a5080b2e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a507f6ae0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ed150 0x559a507f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 --2- 172.21.15.107:0/3220985870 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ec830 0x559a507f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f549f7fe640 0 client.25507 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 >> 172.21.15.107:0/3220985870 conn(0x559a507ea660 msgr2=0x559a507f0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 shutdown_connections 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 1 -- 172.21.15.107:0/3220985870 wait complete. 2024-08-20T06:41:47.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.518+0000 7f54c8a90740 20 client.25507 trim_cache size 0 max 16384 2024-08-20T06:41:47.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.550+0000 7f54c8a90740 1 Processor -- start 2024-08-20T06:41:47.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 -- start start 2024-08-20T06:41:47.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a50810c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 0x559a50811140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507f90e0 0x559a507f3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac010d90 con 0x559a507f90e0 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac00e660 con 0x559a507ec830 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c8a90740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac00c490 con 0x559a507ed150 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54bffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a50810c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54c4c34640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507f90e0 0x559a507f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54bffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a50810c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:36950/0 (socket says 172.21.15.107:36950) 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54bffff640 1 -- 172.21.15.107:0/1140198257 learned_addr learned my addr 172.21.15.107:0/1140198257 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:47.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.551+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 0x559a50811140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1140198257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 msgr2=0x559a50810c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a50810c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1140198257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507f90e0 msgr2=0x559a507f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507f90e0 0x559a507f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 -- 172.21.15.107:0/1140198257 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a507eb0e0 con 0x559a507ed150 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 0x559a50811140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559a507ec7e0 tx=0x7f54b402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18020c3f7cdc6322 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54a5ffb640 1 -- 172.21.15.107:0/1140198257 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40ed040 con 0x559a507ed150 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54a5ffb640 1 -- 172.21.15.107:0/1140198257 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f54b40ca780 con 0x559a507ed150 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.552+0000 7f54a5ffb640 1 -- 172.21.15.107:0/1140198257 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b40cac70 con 0x559a507ed150 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 -- 172.21.15.107:0/1140198257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 msgr2=0x559a50811140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 0x559a50811140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x559a507ec7e0 tx=0x7f54b402a410 comp rx=0 tx=0).stop 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 -- 172.21.15.107:0/1140198257 shutdown_connections 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507f90e0 0x559a507f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507ed150 0x559a50811140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 --2- 172.21.15.107:0/1140198257 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a50810c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 -- 172.21.15.107:0/1140198257 >> 172.21.15.107:0/1140198257 conn(0x559a507ea660 msgr2=0x559a507f7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 -- 172.21.15.107:0/1140198257 shutdown_connections 2024-08-20T06:41:47.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.553+0000 7f54c8a90740 1 -- 172.21.15.107:0/1140198257 wait complete. 2024-08-20T06:41:47.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:47 smithi193 ceph-mon[21944]: pgmap v475: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-08-20T06:41:47.560 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:47 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:47.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.554+0000 7f54c8a90740 1 -- start start 2024-08-20T06:41:47.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.554+0000 7f54c8a90740 1 Processor -- start 2024-08-20T06:41:47.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.554+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.564 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507fd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a50872bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:36966/0 (socket says 172.21.15.107:36966) 2024-08-20T06:41:47.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.565 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bf7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507fd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44764/0 (socket says 172.21.15.107:44764) 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 -- 172.21.15.107:0/2319033961 learned_addr learned my addr 172.21.15.107:0/2319033961 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac00cb20 con 0x559a507ed150 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac00df90 con 0x559a507ec830 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f54ac00d1f0 con 0x559a507f90e0 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c4c34640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a50872bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.566 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 10 client.0 _ll_register_callbacks cb 0x7f54b000b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 -- 172.21.15.107:0/2319033961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 msgr2=0x559a50872bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a50872bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 -- 172.21.15.107:0/2319033961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 msgr2=0x559a507fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a50873110 con 0x559a507ec830 2024-08-20T06:41:47.567 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54b40ca780 con 0x559a507ec830 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f54bffff640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54b0004650 tx=0x7f54b00035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362af3c1a327354b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.555+0000 7f549f7fe640 10 client.25616 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f549f7fe640 1 -- 172.21.15.107:0/2319033961 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0005220 con 0x559a507ec830 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f549f7fe640 1 -- 172.21.15.107:0/2319033961 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f54b00053c0 con 0x559a507ec830 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f549f7fe640 1 -- 172.21.15.107:0/2319033961 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54b00055e0 con 0x559a507ec830 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 10 client.25616 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:47.568 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({fsmap=29}) -- 0x559a508741c0 con 0x559a507ec830 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f549f7fe640 1 -- 172.21.15.107:0/2319033961 <== mon.2 v2:172.21.15.183:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f54b00047a0 con 0x559a507ec830 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 10 client.25616 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 10 client.25616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 20 client.25616 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a508748e0 0x559a50876d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 4 client.25616 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.556+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559a50765830 con 0x559a508748e0 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.557+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a508748e0 0x559a50876d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:47.569 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.557+0000 7f54bf7fe640 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a508748e0 0x559a50876d00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ec9cdcd0aa6bbac server_cookie=507fe6ff382e9be1 in_seq=0 out_seq=0 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.557+0000 7f549f7fe640 10 client.25616 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f549f7fe640 1 -- 172.21.15.107:0/2319033961 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559a50765830 con 0x559a508748e0 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f549f7fe640 10 client.25616 handle_command_reply: tid=0 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 client.25616 shutdown 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 msgr2=0x559a507fd2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f54b0004650 tx=0x7f54b00035f0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 shutdown_connections 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x559a508748e0 0x559a50876d00 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.570 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x559a507f90e0 0x559a50872bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f549f7fe640 0 client.25616 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x559a507ed150 0x559a507fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 --2- 172.21.15.107:0/2319033961 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x559a507ec830 0x559a507fd2c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 >> 172.21.15.107:0/2319033961 conn(0x559a507ea660 msgr2=0x559a507f5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 shutdown_connections 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.558+0000 7f54c8a90740 1 -- 172.21.15.107:0/2319033961 wait complete. 2024-08-20T06:41:47.571 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.559+0000 7f54c8a90740 20 client.25616 trim_cache size 0 max 16384 2024-08-20T06:41:47.572 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:41:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:41:47.572 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:41:47.572 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.559+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 msgr2=0x7f54a80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.559+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 0x7f54a80b6090 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f54b8020000 tx=0x7f54b800ffd0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.559+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 msgr2=0x7f54c010d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.559+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 0x7f54c010d830 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f54b402a930 tx=0x7f54b4003b90 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 shutdown_connections 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f54c0107d60 0x7f54c010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f54c01073e0 0x7f54c010d830 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f54a80b3bd0 0x7f54a80b6090 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 --2- 172.21.15.107:0/1560641895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c01019d0 0x7f54c0116b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:47.573 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 >> 172.21.15.107:0/1560641895 conn(0x7f54c00fd460 msgr2=0x7f54c0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 shutdown_connections 2024-08-20T06:41:47.574 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:47.560+0000 7f54c6697640 1 -- 172.21.15.107:0/1560641895 wait complete. 2024-08-20T06:41:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:47 smithi183 ceph-mon[22204]: pgmap v475: 97 pgs: 97 active+clean; 356 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 355 op/s 2024-08-20T06:41:47.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:47 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:41:48.472 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:41:48.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:41:48.832 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:49.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 -- 172.21.15.107:0/3024531594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5948101990 msgr2=0x7f5948101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:49.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 --2- 172.21.15.107:0/3024531594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5948101990 0x7f5948101d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f593800b9f0 tx=0x7f593802c0a0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 -- 172.21.15.107:0/3024531594 shutdown_connections 2024-08-20T06:41:49.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 --2- 172.21.15.107:0/3024531594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948107d20 0x7f594810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.368 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 --2- 172.21.15.107:0/3024531594 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59481073a0 0x7f59481077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 --2- 172.21.15.107:0/3024531594 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5948101990 0x7f5948101d70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 -- 172.21.15.107:0/3024531594 >> 172.21.15.107:0/3024531594 conn(0x7f59480fd400 msgr2=0x7f59480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:49.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 -- 172.21.15.107:0/3024531594 shutdown_connections 2024-08-20T06:41:49.369 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.367+0000 7f594d707640 1 -- 172.21.15.107:0/3024531594 wait complete. 2024-08-20T06:41:49.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 Processor -- start 2024-08-20T06:41:49.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 -- start start 2024-08-20T06:41:49.370 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 0x7f594810c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 0x7f594810c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5948107d20 0x7f59481a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f594810fd40 con 0x7f5948107d20 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f594810fbc0 con 0x7f59481073a0 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f594d707640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f594810fec0 con 0x7f5948101990 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f5946ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 0x7f594810c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f5946ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 0x7f594810c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:38082/0 (socket says 172.21.15.107:38082) 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f59467fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 0x7f594810c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f5946ffd640 1 -- 172.21.15.107:0/610293745 learned_addr learned my addr 172.21.15.107:0/610293745 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.368+0000 7f59477fe640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5948107d20 0x7f59481a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 -- 172.21.15.107:0/610293745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 msgr2=0x7f594810c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 0x7f594810c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 -- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5948107d20 msgr2=0x7f59481a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:49.371 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5948107d20 0x7f59481a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 -- 172.21.15.107:0/610293745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59481a33a0 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f59467fc640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 0x7f594810c8b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f593c00e790 tx=0x7f593c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c00eea0 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f593c00ccc0 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f593c018680 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59481a3690 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.369+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481abfb0 con 0x7f59481073a0 2024-08-20T06:41:49.372 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.371+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5948101d70 con 0x7f59481073a0 2024-08-20T06:41:49.373 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.372+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f593c005070 con 0x7f59481073a0 2024-08-20T06:41:49.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.373+0000 7f592ffff640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 0x7f59300b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:49.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.373+0000 7f5946ffd640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 0x7f59300b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:49.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.373+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f593c0e3cf0 con 0x7f59481073a0 2024-08-20T06:41:49.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.374+0000 7f5946ffd640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 0x7f59300b6110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5938005cd0 tx=0x7f593802e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:49.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.375+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f593c0aacf0 con 0x7f59481073a0 2024-08-20T06:41:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:49 smithi107 ceph-mon[18495]: pgmap v476: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-08-20T06:41:49.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:49 smithi183 ceph-mon[22204]: pgmap v476: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-08-20T06:41:49.768 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.767+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5948111c30 con 0x7f59481073a0 2024-08-20T06:41:49.769 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.768+0000 7f592ffff640 1 -- 172.21.15.107:0/610293745 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f593c0af240 con 0x7f59481073a0 2024-08-20T06:41:49.769 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:41:49.769 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"btime":"2024-08-20T06:41:46:295359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:41:45.321550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:41:49.770 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-08-20T06:41:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 msgr2=0x7f59300b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 0x7f59300b6110 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5938005cd0 tx=0x7f593802e040 comp rx=0 tx=0).stop 2024-08-20T06:41:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 msgr2=0x7f594810c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:49.773 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 0x7f594810c8b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f593c00e790 tx=0x7f593c00ec50 comp rx=0 tx=0).stop 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 shutdown_connections 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5948107d20 0x7f59481a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f59481073a0 0x7f594810c8b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f59300b3c50 0x7f59300b6110 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 --2- 172.21.15.107:0/610293745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5948101990 0x7f594810c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 >> 172.21.15.107:0/610293745 conn(0x7f59480fd400 msgr2=0x7f59480ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 shutdown_connections 2024-08-20T06:41:49.774 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:49.772+0000 7f594d707640 1 -- 172.21.15.107:0/610293745 wait complete. 2024-08-20T06:41:49.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:49 smithi193 ceph-mon[21944]: pgmap v476: 97 pgs: 97 active+clean; 350 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.2 MiB/s wr, 352 op/s 2024-08-20T06:41:50.606 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:50 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/610293745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:50.643 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:41:50.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:50 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/610293745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:50.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:50 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/610293745' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:41:50.994 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:51.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:51 smithi107 ceph-mon[18495]: pgmap v477: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-08-20T06:41:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 -- 172.21.15.107:0/1643508921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 msgr2=0x7f00b8101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 --2- 172.21.15.107:0/1643508921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8101d70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009900 tx=0x7f00a802a410 comp rx=0 tx=0).stop 2024-08-20T06:41:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 -- 172.21.15.107:0/1643508921 shutdown_connections 2024-08-20T06:41:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 --2- 172.21.15.107:0/1643508921 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b8107d20 0x7f00b810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 --2- 172.21.15.107:0/1643508921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b81073a0 0x7f00b81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 --2- 172.21.15.107:0/1643508921 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8101d70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 -- 172.21.15.107:0/1643508921 >> 172.21.15.107:0/1643508921 conn(0x7f00b80fd400 msgr2=0x7f00b80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 -- 172.21.15.107:0/1643508921 shutdown_connections 2024-08-20T06:41:51.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.511+0000 7f00bcd39640 1 -- 172.21.15.107:0/1643508921 wait complete. 2024-08-20T06:41:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.512+0000 7f00bcd39640 1 Processor -- start 2024-08-20T06:41:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.512+0000 7f00bcd39640 1 -- start start 2024-08-20T06:41:51.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.512+0000 7f00bcd39640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.512+0000 7f00bcd39640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 0x7f00b8199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00bcd39640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b8107d20 0x7f00b819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44804/0 (socket says 172.21.15.107:44804) 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00bcd39640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8112a50 con 0x7f00b8101990 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00bcd39640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00b806b680 con 0x7f00b81073a0 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b659c640 1 -- 172.21.15.107:0/2061041480 learned_addr learned my addr 172.21.15.107:0/2061041480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8112bd0 con 0x7f00b8107d20 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 0x7f00b8199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b8107d20 0x7f00b819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 -- 172.21.15.107:0/2061041480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b8107d20 msgr2=0x7f00b819e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b8107d20 0x7f00b819e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.515 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 -- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 msgr2=0x7f00b8198f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b5d9b640 1 -- 172.21.15.107:0/2061041480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b819f000 con 0x7f00b81073a0 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.513+0000 7f00b659c640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 0x7f00b8199490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00a77fe640 1 -- 172.21.15.107:0/2061041480 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac009c30 con 0x7f00b81073a0 2024-08-20T06:41:51.516 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00a77fe640 1 -- 172.21.15.107:0/2061041480 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00ac0044c0 con 0x7f00b81073a0 2024-08-20T06:41:51.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b819f2f0 con 0x7f00b81073a0 2024-08-20T06:41:51.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00a77fe640 1 -- 172.21.15.107:0/2061041480 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac003db0 con 0x7f00b81073a0 2024-08-20T06:41:51.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.514+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81a59b0 con 0x7f00b81073a0 2024-08-20T06:41:51.517 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.517+0000 7f00a77fe640 1 -- 172.21.15.107:0/2061041480 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f00ac003f50 con 0x7f00b81073a0 2024-08-20T06:41:51.518 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.517+0000 7f00bf132740 1 Processor -- start 2024-08-20T06:41:51.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.517+0000 7f00bf132740 1 -- start start 2024-08-20T06:41:51.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00a77fe640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 0x7f00a00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.519 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d1391a0 0x55c74d144bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d145130 0x55c74d147520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c74d0af930 con 0x55c74d145130 2024-08-20T06:41:51.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c74d09ee30 con 0x55c74d1391a0 2024-08-20T06:41:51.520 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00bf132740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c74d0a2e70 con 0x55c74d138880 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00b659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00b659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:38120/0 (socket says 172.21.15.107:38120) 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00b659c640 1 -- 172.21.15.107:0/3560853659 learned_addr learned my addr 172.21.15.107:0/3560853659 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00a77fe640 1 -- 172.21.15.107:0/2061041480 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f00ac0e39d0 con 0x7f00b81073a0 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.518+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d1391a0 0x55c74d144bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 -- 172.21.15.107:0/3560853659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d1391a0 msgr2=0x55c74d144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d1391a0 0x55c74d144bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 -- 172.21.15.107:0/3560853659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d145130 msgr2=0x55c74d147520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.521 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d145130 0x55c74d147520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 -- 172.21.15.107:0/3560853659 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74cdfe650 con 0x55c74d138880 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 0x7f00a00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00b659c640 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a8032f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7dcd1e78541e363 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00a4ff9640 1 -- 172.21.15.107:0/3560853659 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8035070 con 0x55c74d138880 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00a4ff9640 1 -- 172.21.15.107:0/3560853659 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00a80044c0 con 0x55c74d138880 2024-08-20T06:41:51.522 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.519+0000 7f00a4ff9640 1 -- 172.21.15.107:0/3560853659 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8003df0 con 0x55c74d138880 2024-08-20T06:41:51.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 0x7f00a00b6090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55c74d138830 tx=0x7f00b0009c10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 -- 172.21.15.107:0/3560853659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 msgr2=0x55c74d138c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.524 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006010 tx=0x7f00a8032f70 comp rx=0 tx=0).stop 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 -- 172.21.15.107:0/3560853659 shutdown_connections 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d145130 0x55c74d147520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d1391a0 0x55c74d144bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 --2- 172.21.15.107:0/3560853659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d138c60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 -- 172.21.15.107:0/3560853659 >> 172.21.15.107:0/3560853659 conn(0x55c74d1366b0 msgr2=0x55c74d137850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.520+0000 7f00bf132740 1 -- 172.21.15.107:0/3560853659 shutdown_connections 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.521+0000 7f00bf132740 1 -- 172.21.15.107:0/3560853659 wait complete. 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.521+0000 7f00bf132740 1 -- start start 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.521+0000 7f00bf132740 1 Processor -- start 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.521+0000 7f00bf132740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d138880 0x55c74d1420c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.525 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 0x55c74d142b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55c74cf01700 con 0x55c74d1391a0 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55c74d025810 con 0x55c74d138880 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55c74d147e20 con 0x55c74d145130 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00bf132740 10 client.0 _ll_register_callbacks cb 0x55c74d098700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b5d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44832/0 (socket says 172.21.15.107:44832) 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b5d9b640 1 -- 172.21.15.107:0/66335327 learned_addr learned my addr 172.21.15.107:0/66335327 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b659c640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d138880 0x55c74d1420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 0x55c74d142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 -- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d138880 msgr2=0x55c74d1420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.526 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d138880 0x55c74d1420c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 -- 172.21.15.107:0/66335327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 msgr2=0x55c74d142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b6d9d640 1 -- 172.21.15.107:0/66335327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74d1555e0 con 0x55c74d145130 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.522+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 0x55c74d142b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00b00052f0 tx=0x7f00b0009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=580767ccbc434927 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f009dffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000eb30 con 0x55c74d145130 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f00b000ecd0 con 0x55c74d145130 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c74d04e150 con 0x55c74d145130 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00b001e420 con 0x55c74d145130 2024-08-20T06:41:51.527 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f00bf132740 10 client.25528 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:51.528 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.523+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c74d157ff0 con 0x55c74d145130 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c74d04e150 con 0x55c74d145130 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f00b00305c0 con 0x55c74d145130 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 10 client.25528 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 10 client.25528 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 20 client.25528 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d159400 0x55c74d15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 4 client.25528 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c74d0b1880 con 0x55c74d159400 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.524+0000 7f00b659c640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d159400 0x55c74d15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.525+0000 7f00b659c640 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d159400 0x55c74d15b7f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54e1de591e132e64 server_cookie=913e79fcf47dda in_seq=0 out_seq=0 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.525+0000 7f009dffb640 10 client.25528 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.525+0000 7f009dffb640 1 -- 172.21.15.107:0/66335327 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c74d0b1880 con 0x55c74d159400 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.525+0000 7f009dffb640 10 client.25528 handle_command_reply: tid=0 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.525+0000 7f00bf132740 1 client.25528 shutdown 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 msgr2=0x55c74d142b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 0x55c74d142b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f00b00052f0 tx=0x7f00b0009500 comp rx=0 tx=0).stop 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 shutdown_connections 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d145130 0x55c74d142b40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d159400 0x55c74d15b7f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 --2- 172.21.15.107:0/66335327 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d138880 0x55c74d1420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f009dffb640 0 client.25528 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 >> 172.21.15.107:0/66335327 conn(0x55c74d1366b0 msgr2=0x55c74d13c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 shutdown_connections 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 1 -- 172.21.15.107:0/66335327 wait complete. 2024-08-20T06:41:51.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.526+0000 7f00bf132740 20 client.25528 trim_cache size 0 max 16384 2024-08-20T06:41:51.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 Processor -- start 2024-08-20T06:41:51.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 -- start start 2024-08-20T06:41:51.547 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d13c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ae890 con 0x55c74d1391a0 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00b659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00b659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d13c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:38142/0 (socket says 172.21.15.107:38142) 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00b659c640 1 -- 172.21.15.107:0/3676533784 learned_addr learned my addr 172.21.15.107:0/3676533784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81ac160 con 0x55c74d145130 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.545+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81a9f90 con 0x55c74d138880 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 -- 172.21.15.107:0/3676533784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 msgr2=0x55c74d13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.548 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d13c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 -- 172.21.15.107:0/3676533784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 msgr2=0x55c74d1425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d1425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 -- 172.21.15.107:0/3676533784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c74d137130 con 0x55c74d145130 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d142b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00b0030ad0 tx=0x7f00b00049c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4fb53aa4c8532b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f009ffff640 1 -- 172.21.15.107:0/3676533784 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0030b50 con 0x55c74d145130 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f009ffff640 1 -- 172.21.15.107:0/3676533784 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f00b0030cf0 con 0x55c74d145130 2024-08-20T06:41:51.549 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.546+0000 7f009ffff640 1 -- 172.21.15.107:0/3676533784 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b001e820 con 0x55c74d145130 2024-08-20T06:41:51.551 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.547+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 msgr2=0x55c74d142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.547+0000 7f00bf132740 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d142b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f00b0030ad0 tx=0x7f00b00049c0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 shutdown_connections 2024-08-20T06:41:51.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d142b00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d1391a0 0x55c74d1425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 --2- 172.21.15.107:0/3676533784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d138880 0x55c74d13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 >> 172.21.15.107:0/3676533784 conn(0x55c74d1366b0 msgr2=0x55c74d13d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 shutdown_connections 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 -- 172.21.15.107:0/3676533784 wait complete. 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 -- start start 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.548+0000 7f00bf132740 1 Processor -- start 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d138880 0x55c74d13f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d1391a0 0x55c74d14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81aa620 con 0x55c74d138880 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81aba90 con 0x55c74d145130 2024-08-20T06:41:51.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81aacf0 con 0x55c74d1391a0 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00bf132740 10 client.0 _ll_register_callbacks cb 0x7f00b0009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:37048/0 (socket says 172.21.15.107:37048) 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 -- 172.21.15.107:0/4099619424 learned_addr learned my addr 172.21.15.107:0/4099619424 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b659c640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d138880 0x55c74d13f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b5d9b640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d1391a0 0x55c74d14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 -- 172.21.15.107:0/4099619424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d1391a0 msgr2=0x55c74d14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d1391a0 0x55c74d14d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 -- 172.21.15.107:0/4099619424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d138880 msgr2=0x55c74d13f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d138880 0x55c74d13f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 -- 172.21.15.107:0/4099619424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c74d14dce0 con 0x55c74d145130 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f00b6d9d640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009540 tx=0x7f00b000e9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b6d26f9e8a48de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.549+0000 7f009dffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0049060 con 0x55c74d145130 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f00b001e5c0 con 0x55c74d145130 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c74d0ad320 con 0x55c74d145130 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f00b0030b50 con 0x55c74d145130 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f00bf132740 10 client.25640 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:51.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c74d1c3150 con 0x55c74d145130 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.550+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55c74d0ad320 con 0x55c74d145130 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f00b0044780 con 0x55c74d145130 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 10 client.25640 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 10 client.25640 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 20 client.25640 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d1c36f0 0x55c74d1c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 4 client.25640 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c74d0aaa60 con 0x55c74d1c36f0 2024-08-20T06:41:51.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00b659c640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d1c36f0 0x55c74d1c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f009dffb640 10 client.25640 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.551+0000 7f00b659c640 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d1c36f0 0x55c74d1c5b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a98181254487fec1 server_cookie=3abbacaf839e78c6 in_seq=0 out_seq=0 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f009dffb640 1 -- 172.21.15.107:0/4099619424 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c74d0aaa60 con 0x55c74d1c36f0 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f009dffb640 10 client.25640 handle_command_reply: tid=0 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 client.25640 shutdown 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 msgr2=0x55c74d14d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009540 tx=0x7f00b000e9e0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 shutdown_connections 2024-08-20T06:41:51.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55c74d1c36f0 0x55c74d1c5b10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55c74d145130 0x55c74d14d5f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f009dffb640 0 client.25640 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c74d1391a0 0x55c74d14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 --2- 172.21.15.107:0/4099619424 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55c74d138880 0x55c74d13f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 >> 172.21.15.107:0/4099619424 conn(0x55c74d1366b0 msgr2=0x55c74d1436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 shutdown_connections 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 1 -- 172.21.15.107:0/4099619424 wait complete. 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.552+0000 7f00bf132740 20 client.25640 trim_cache size 0 max 16384 2024-08-20T06:41:51.559 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:51.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 msgr2=0x7f00a00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.560 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 0x7f00a00b6090 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55c74d138830 tx=0x7f00b0009c10 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 msgr2=0x7f00b8199490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 0x7f00b8199490 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f00ac005fd0 tx=0x7f00ac00dde0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 shutdown_connections 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f00a00b3bd0 0x7f00a00b6090 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00b8107d20 0x7f00b819e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f00b81073a0 0x7f00b8199490 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 --2- 172.21.15.107:0/2061041480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f00b8101990 0x7f00b8198f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 >> 172.21.15.107:0/2061041480 conn(0x7f00b80fd400 msgr2=0x7f00b80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 shutdown_connections 2024-08-20T06:41:51.561 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:51.553+0000 7f00bcd39640 1 -- 172.21.15.107:0/2061041480 wait complete. 2024-08-20T06:41:51.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:51 smithi183 ceph-mon[22204]: pgmap v477: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-08-20T06:41:51.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:51 smithi193 ceph-mon[21944]: pgmap v477: 97 pgs: 97 active+clean; 351 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-08-20T06:41:52.492 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:52 smithi107 ceph-mon[18495]: pgmap v478: 97 pgs: 97 active+clean; 357 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-08-20T06:41:52.531 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:41:52.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:52 smithi183 ceph-mon[22204]: pgmap v478: 97 pgs: 97 active+clean; 357 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-08-20T06:41:52.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:52 smithi193 ceph-mon[21944]: pgmap v478: 97 pgs: 97 active+clean; 357 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 332 op/s 2024-08-20T06:41:52.901 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:53.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.449+0000 7f12b0f2a640 1 -- 172.21.15.107:0/3422403997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075c70 msgr2=0x7f12ac10eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.449+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/3422403997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075c70 0x7f12ac10eaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009860 tx=0x7f12a002a340 comp rx=0 tx=0).stop 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 -- 172.21.15.107:0/3422403997 shutdown_connections 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/3422403997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac10f0c0 0x7f12ac1114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/3422403997 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075c70 0x7f12ac10eaf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/3422403997 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac075270 0x7f12ac075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 -- 172.21.15.107:0/3422403997 >> 172.21.15.107:0/3422403997 conn(0x7f12ac0fe550 msgr2=0x7f12ac100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 -- 172.21.15.107:0/3422403997 shutdown_connections 2024-08-20T06:41:53.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.450+0000 7f12b0f2a640 1 -- 172.21.15.107:0/3422403997 wait complete. 2024-08-20T06:41:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.451+0000 7f12b0f2a640 1 Processor -- start 2024-08-20T06:41:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.451+0000 7f12b0f2a640 1 -- start start 2024-08-20T06:41:53.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.451+0000 7f12b0f2a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075270 0x7f12ac19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.451+0000 7f12b0f2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.451+0000 7f12b0f2a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac10f0c0 0x7f12ac1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12b0f2a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac114b70 con 0x7f12ac10f0c0 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12b0f2a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1149f0 con 0x7f12ac075270 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12b0f2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac114cf0 con 0x7f12ac075c70 2024-08-20T06:41:53.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075270 0x7f12ac19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac10f0c0 0x7f12ac1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:38178/0 (socket says 172.21.15.107:38178) 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1123340081 learned_addr learned my addr 172.21.15.107:0/1123340081 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1123340081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075270 msgr2=0x7f12ac19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075270 0x7f12ac19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac10f0c0 msgr2=0x7f12ac1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac10f0c0 0x7f12ac1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1123340081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac1a11d0 con 0x7f12ac075c70 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.452+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000c00 tx=0x7f12a002ae50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.453+0000 7f129b7fe640 1 -- 172.21.15.107:0/1123340081 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a0037040 con 0x7f12ac075c70 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.453+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac1a1460 con 0x7f12ac075c70 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.453+0000 7f129b7fe640 1 -- 172.21.15.107:0/1123340081 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12a0045030 con 0x7f12ac075c70 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.453+0000 7f129b7fe640 1 -- 172.21.15.107:0/1123340081 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a00305a0 con 0x7f12ac075c70 2024-08-20T06:41:53.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.453+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac102680 con 0x7f12ac075c70 2024-08-20T06:41:53.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.456+0000 7f129b7fe640 1 -- 172.21.15.107:0/1123340081 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f12a0004050 con 0x7f12ac075c70 2024-08-20T06:41:53.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f129b7fe640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 0x7f12940b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 Processor -- start 2024-08-20T06:41:53.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12aa59c640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 0x7f12940b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f129b7fe640 1 -- 172.21.15.107:0/1123340081 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f12a010b8c0 con 0x7f12ac075c70 2024-08-20T06:41:53.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 -- start start 2024-08-20T06:41:53.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 0x55d7f6884bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6887520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7f67ef930 con 0x55d7f6878880 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7f67dee30 con 0x55d7f68791a0 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12b3323740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7f67e2e70 con 0x55d7f6885130 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44874/0 (socket says 172.21.15.107:44874) 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.457+0000 7f12aa59c640 1 -- 172.21.15.107:0/186868156 learned_addr learned my addr 172.21.15.107:0/186868156 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 0x55d7f6884bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12aa59c640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 0x7f12940b6150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f129c009530 tx=0x7f129c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12aad9d640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6887520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 -- 172.21.15.107:0/186868156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 msgr2=0x55d7f6887520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6887520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 -- 172.21.15.107:0/186868156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 msgr2=0x55d7f6878c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.461 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 -- 172.21.15.107:0/186868156 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f653e650 con 0x55d7f68791a0 2024-08-20T06:41:53.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12aa59c640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:53.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 0x55d7f6884bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12a00d28a0 tx=0x7f12a002a840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2adb5b75a3363fb9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f1298ff9640 1 -- 172.21.15.107:0/186868156 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a004f030 con 0x55d7f68791a0 2024-08-20T06:41:53.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f1298ff9640 1 -- 172.21.15.107:0/186868156 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f12a00d2420 con 0x55d7f68791a0 2024-08-20T06:41:53.462 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.458+0000 7f1298ff9640 1 -- 172.21.15.107:0/186868156 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a00db880 con 0x55d7f68791a0 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.459+0000 7f12b3323740 1 -- 172.21.15.107:0/186868156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 msgr2=0x55d7f6884bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.459+0000 7f12b3323740 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 0x55d7f6884bf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12a00d28a0 tx=0x7f12a002a840 comp rx=0 tx=0).stop 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 -- 172.21.15.107:0/186868156 shutdown_connections 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6887520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f68791a0 0x55d7f6884bf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 --2- 172.21.15.107:0/186868156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6878c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 -- 172.21.15.107:0/186868156 >> 172.21.15.107:0/186868156 conn(0x55d7f68766b0 msgr2=0x55d7f6877850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 -- 172.21.15.107:0/186868156 shutdown_connections 2024-08-20T06:41:53.464 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 -- 172.21.15.107:0/186868156 wait complete. 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 -- start start 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.460+0000 7f12b3323740 1 Processor -- start 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6882b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d7f6641700 con 0x55d7f68791a0 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d7f6765810 con 0x55d7f6878880 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d7f6887ba0 con 0x55d7f6885130 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44890/0 (socket says 172.21.15.107:44890) 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1527206811 learned_addr learned my addr 172.21.15.107:0/1527206811 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:53.465 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12b3323740 10 client.0 _ll_register_callbacks cb 0x55d7f67d8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12aad9d640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6882b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 msgr2=0x55d7f6882b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6882b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1527206811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 msgr2=0x55d7f68820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68820b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12a9d9b640 1 -- 172.21.15.107:0/1527206811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f6890e40 con 0x55d7f68791a0 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12aad9d640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6882b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:53.466 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.461+0000 7f12aa59c640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.462+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f12a0035040 tx=0x7f12a0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6ba519a29c82e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.462+0000 7f1291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.462+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a004f070 con 0x55d7f68791a0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.462+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f12a00db040 con 0x55d7f68791a0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d7f678e150 con 0x55d7f68791a0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12a00d2910 con 0x55d7f68791a0 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 10 client.16134 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:53.467 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7f68937d0 con 0x55d7f68791a0 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55d7f678e150 con 0x55d7f68791a0 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f12a00d3b90 con 0x55d7f68791a0 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 10 client.16134 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 10 client.16134 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 20 client.16134 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6896f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 4 client.16134 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.463+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7f67f1880 con 0x55d7f6894b20 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.464+0000 7f12aa59c640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6896f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.464+0000 7f1291ffb640 10 client.16134 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.464+0000 7f12aa59c640 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6896f10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b25514cf9640f698 server_cookie=cd37937dc28d3dc4 in_seq=0 out_seq=0 2024-08-20T06:41:53.469 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f1291ffb640 1 -- 172.21.15.107:0/1527206811 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d7f67f1880 con 0x55d7f6894b20 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f1291ffb640 10 client.16134 handle_command_reply: tid=0 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 client.16134 shutdown 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 msgr2=0x55d7f68825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f12a0035040 tx=0x7f12a0004610 comp rx=0 tx=0).stop 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 shutdown_connections 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6896f10 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f1291ffb640 0 client.16134 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f6885130 0x55d7f6882b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f68791a0 0x55d7f68825f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 --2- 172.21.15.107:0/1527206811 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.465+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 >> 172.21.15.107:0/1527206811 conn(0x55d7f68766b0 msgr2=0x55d7f687c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.466+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 shutdown_connections 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.466+0000 7f12b3323740 1 -- 172.21.15.107:0/1527206811 wait complete. 2024-08-20T06:41:53.470 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.466+0000 7f12b3323740 20 client.16134 trim_cache size 0 max 16384 2024-08-20T06:41:53.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.482+0000 7f12b3323740 1 Processor -- start 2024-08-20T06:41:53.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.482+0000 7f12b3323740 1 -- start start 2024-08-20T06:41:53.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.482+0000 7f12b3323740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.489 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12b3323740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f687c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12b3323740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 0x55d7f6883df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12b3323740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1ae7a0 con 0x55d7f6878880 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44906/0 (socket says 172.21.15.107:44906) 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 -- 172.21.15.107:0/4188607337 learned_addr learned my addr 172.21.15.107:0/4188607337 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1ac070 con 0x55d7f6885130 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1a9f00 con 0x55d7f68791a0 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f687c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aad9d640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 0x55d7f6883df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 -- 172.21.15.107:0/4188607337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 msgr2=0x55d7f687c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.490 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f687c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 -- 172.21.15.107:0/4188607337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 msgr2=0x55d7f6883df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 0x55d7f6883df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 -- 172.21.15.107:0/4188607337 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7f6877130 con 0x55d7f6878880 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aad9d640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 0x55d7f6883df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12a9d9b640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f687c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.483+0000 7f12aa59c640 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f129c005ab0 tx=0x7f129c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3589906ce2173a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.484+0000 7f1293fff640 1 -- 172.21.15.107:0/4188607337 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c00ecc0 con 0x55d7f6878880 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.484+0000 7f1293fff640 1 -- 172.21.15.107:0/4188607337 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f129c0054b0 con 0x55d7f6878880 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.484+0000 7f1293fff640 1 -- 172.21.15.107:0/4188607337 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c004e10 con 0x55d7f6878880 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 msgr2=0x55d7f6877d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f129c005ab0 tx=0x7f129c014000 comp rx=0 tx=0).stop 2024-08-20T06:41:53.491 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 shutdown_connections 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6885130 0x55d7f6883df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f687c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- 172.21.15.107:0/4188607337 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6878880 0x55d7f6877d70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 >> 172.21.15.107:0/4188607337 conn(0x55d7f68766b0 msgr2=0x55d7f68836e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 shutdown_connections 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- 172.21.15.107:0/4188607337 wait complete. 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 -- start start 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 Processor -- start 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.492 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f68f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.485+0000 7f12b3323740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12b3323740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1aa550 con 0x55d7f6885130 2024-08-20T06:41:53.497 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12b3323740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1ab9a0 con 0x55d7f6878880 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12b3323740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac1aac00 con 0x55d7f68791a0 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44914/0 (socket says 172.21.15.107:44914) 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 -- 172.21.15.107:0/810723492 learned_addr learned my addr 172.21.15.107:0/810723492 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aa59c640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12b3323740 10 client.0 _ll_register_callbacks cb 0x7f129c009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12a9d9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f68f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 -- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 msgr2=0x55d7f68f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f68f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 -- 172.21.15.107:0/810723492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 msgr2=0x55d7f68f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:53.498 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 -- 172.21.15.107:0/810723492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7f6890bf0 con 0x55d7f6885130 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aa59c640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f12aad9d640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f12a4003fa0 tx=0x7f12a40044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b1666afcb061c50 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f1291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a40128d0 con 0x55d7f6885130 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.486+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f12a4012a70 con 0x55d7f6885130 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f129c0054b0 con 0x55d7f6885130 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12a4012c70 con 0x55d7f6885130 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 10 client.16140 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:53.499 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d7f68ff040 con 0x55d7f6885130 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f129c0054b0 con 0x55d7f6885130 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f12a4026020 con 0x55d7f6885130 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 10 client.16140 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 10 client.16140 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 20 client.16140 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:53.500 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6901860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 4 client.16140 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.487+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d7f67ed320 con 0x55d7f6894b20 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12aa59c640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6901860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f1291ffb640 10 client.16140 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12aa59c640 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6901860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e134716aef36016c server_cookie=889160d05e930566 in_seq=0 out_seq=0 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f1291ffb640 1 -- 172.21.15.107:0/810723492 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d7f67ed320 con 0x55d7f6894b20 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f1291ffb640 10 client.16140 handle_command_reply: tid=0 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 client.16140 shutdown 2024-08-20T06:41:53.501 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 msgr2=0x55d7f688ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f12a4003fa0 tx=0x7f12a40044d0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 shutdown_connections 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x55d7f6894b20 0x55d7f6901860 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d7f6885130 0x55d7f688ca80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7f68791a0 0x55d7f68f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f1291ffb640 0 client.16140 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 --2- 172.21.15.107:0/810723492 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d7f6878880 0x55d7f68f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.488+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 >> 172.21.15.107:0/810723492 conn(0x55d7f68766b0 msgr2=0x55d7f6890400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 shutdown_connections 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b3323740 1 -- 172.21.15.107:0/810723492 wait complete. 2024-08-20T06:41:53.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b3323740 20 client.16140 trim_cache size 0 max 16384 2024-08-20T06:41:53.503 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 msgr2=0x7f12940b6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 0x7f12940b6150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f129c009530 tx=0x7f129c009260 comp rx=0 tx=0).stop 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 msgr2=0x7f12ac19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.489+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f12a0000c00 tx=0x7f12a002ae50 comp rx=0 tx=0).stop 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 shutdown_connections 2024-08-20T06:41:53.504 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f12ac10f0c0 0x7f12ac1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f12940b3c90 0x7f12940b6150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12ac075c70 0x7f12ac19b660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 --2- 172.21.15.107:0/1123340081 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f12ac075270 0x7f12ac19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 >> 172.21.15.107:0/1123340081 conn(0x7f12ac0fe550 msgr2=0x7f12ac0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 shutdown_connections 2024-08-20T06:41:53.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:53.490+0000 7f12b0f2a640 1 -- 172.21.15.107:0/1123340081 wait complete. 2024-08-20T06:41:54.506 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:41:54.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:54 smithi183 ceph-mon[22204]: pgmap v479: 97 pgs: 97 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-08-20T06:41:54.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:54 smithi193 ceph-mon[21944]: pgmap v479: 97 pgs: 97 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-08-20T06:41:54.857 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:54.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:54 smithi107 ceph-mon[18495]: pgmap v479: 97 pgs: 97 active+clean; 355 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.6 MiB/s wr, 360 op/s 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 -- 172.21.15.107:0/3572580578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c1042c0 msgr2=0x7f6a8c10c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 --2- 172.21.15.107:0/3572580578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c10c7b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f6a7800fb10 tx=0x7f6a7802e570 comp rx=0 tx=0).stop 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 -- 172.21.15.107:0/3572580578 shutdown_connections 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 --2- 172.21.15.107:0/3572580578 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c10ce00 0x7f6a8c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 --2- 172.21.15.107:0/3572580578 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c10c7b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 --2- 172.21.15.107:0/3572580578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.402+0000 7f6a9315e640 1 -- 172.21.15.107:0/3572580578 >> 172.21.15.107:0/3572580578 conn(0x7f6a8c0fd400 msgr2=0x7f6a8c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.403+0000 7f6a9315e640 1 -- 172.21.15.107:0/3572580578 shutdown_connections 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.403+0000 7f6a9315e640 1 -- 172.21.15.107:0/3572580578 wait complete. 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 Processor -- start 2024-08-20T06:41:55.409 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 -- start start 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c10ce00 0x7f6a8c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c110790 con 0x7f6a8c10ce00 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c110610 con 0x7f6a8c1039a0 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a9315e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c110910 con 0x7f6a8c1042c0 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a90efa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a90efa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44682/0 (socket says 172.21.15.107:44682) 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a90efa640 1 -- 172.21.15.107:0/946164269 learned_addr learned my addr 172.21.15.107:0/946164269 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.404+0000 7f6a916fb640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c10ce00 0x7f6a8c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 -- 172.21.15.107:0/946164269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1042c0 msgr2=0x7f6a8c197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.410 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 -- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c10ce00 msgr2=0x7f6a8c19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c10ce00 0x7f6a8c19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 -- 172.21.15.107:0/946164269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8c19cb80 con 0x7f6a8c1039a0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a90efa640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000e9f0 tx=0x7f6a8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a89ffb640 1 -- 172.21.15.107:0/946164269 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8000cd80 con 0x7f6a8c1039a0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a89ffb640 1 -- 172.21.15.107:0/946164269 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a800108c0 con 0x7f6a8c1039a0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a89ffb640 1 -- 172.21.15.107:0/946164269 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a800187c0 con 0x7f6a8c1039a0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.405+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8c19ce70 con 0x7f6a8c1039a0 2024-08-20T06:41:55.411 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.406+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a8c1a3530 con 0x7f6a8c1039a0 2024-08-20T06:41:55.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.410+0000 7f6a95557740 1 Processor -- start 2024-08-20T06:41:55.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.410+0000 7f6a89ffb640 1 -- 172.21.15.107:0/946164269 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f6a800189a0 con 0x7f6a8c1039a0 2024-08-20T06:41:55.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.411+0000 7f6a89ffb640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 0x7f6a7c0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.411+0000 7f6a89ffb640 1 -- 172.21.15.107:0/946164269 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f6a800e3fb0 con 0x7f6a8c1039a0 2024-08-20T06:41:55.413 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.412+0000 7f6a95557740 1 -- start start 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x557740371320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5577402df930 con 0x557740371860 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5577402cee30 con 0x55774037d7f0 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a95557740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577402d2e70 con 0x557740370f40 2024-08-20T06:41:55.414 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a916fb640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 0x7f6a7c0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58394/0 (socket says 172.21.15.107:58394) 2024-08-20T06:41:55.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a8bfff640 1 -- 172.21.15.107:0/875945478 learned_addr learned my addr 172.21.15.107:0/875945478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:55.415 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.413+0000 7f6a916fb640 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.414+0000 7f6a916fb640 1 -- 172.21.15.107:0/875945478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 msgr2=0x557740371320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T06:41:55.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.414+0000 7f6a916fb640 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x557740371320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.414+0000 7f6a916fb640 1 -- 172.21.15.107:0/875945478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 msgr2=0x55774037d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.416 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.414+0000 7f6a916fb640 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.414+0000 7f6a916fb640 1 -- 172.21.15.107:0/875945478 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774002e650 con 0x55774037d7f0 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.415+0000 7f6a916fb640 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037fbe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e00 tx=0x7f6a8401d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0739bc6aa2563a8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.415+0000 7f6a916fb640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 0x7f6a7c0b61d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005fd0 tx=0x7f6a84009b70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.416+0000 7f6a76ffd640 1 -- 172.21.15.107:0/875945478 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84013de0 con 0x55774037d7f0 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.416+0000 7f6a76ffd640 1 -- 172.21.15.107:0/875945478 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a84013170 con 0x55774037d7f0 2024-08-20T06:41:55.417 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.416+0000 7f6a76ffd640 1 -- 172.21.15.107:0/875945478 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84003cf0 con 0x55774037d7f0 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 -- 172.21.15.107:0/875945478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 msgr2=0x55774037fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037fbe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009e00 tx=0x7f6a8401d000 comp rx=0 tx=0).stop 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 -- 172.21.15.107:0/875945478 shutdown_connections 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037fbe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 --2- 172.21.15.107:0/875945478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x557740371320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.418 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 -- 172.21.15.107:0/875945478 >> 172.21.15.107:0/875945478 conn(0x55774036ed70 msgr2=0x55774036ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 -- 172.21.15.107:0/875945478 shutdown_connections 2024-08-20T06:41:55.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.417+0000 7f6a95557740 1 -- 172.21.15.107:0/875945478 wait complete. 2024-08-20T06:41:55.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.418+0000 7f6a95557740 1 -- start start 2024-08-20T06:41:55.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.418+0000 7f6a95557740 1 Processor -- start 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 0x55774037a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x557740131700 con 0x557740370f40 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x557740255810 con 0x55774037d7f0 2024-08-20T06:41:55.420 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5577403803b0 con 0x557740371860 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a95557740 10 client.0 _ll_register_callbacks cb 0x5577402c8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a90efa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 0x55774037a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a90efa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 0x55774037a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58402/0 (socket says 172.21.15.107:58402) 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:57560/0 (socket says 172.21.15.107:57560) 2024-08-20T06:41:55.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a90efa640 1 -- 172.21.15.107:0/3427041285 learned_addr learned my addr 172.21.15.107:0/3427041285 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 -- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 msgr2=0x55774037b300 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 -- 172.21.15.107:0/3427041285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 msgr2=0x55774037a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 0x55774037a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.419+0000 7f6a8bfff640 1 -- 172.21.15.107:0/3427041285 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577403898b0 con 0x557740371860 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.420+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005ab0 tx=0x7f6a7800eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6a1aa60e4ed5981 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.420+0000 7f6a6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.420+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7800f720 con 0x557740371860 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.420+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f6a78004030 con 0x557740371860 2024-08-20T06:41:55.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.420+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55774027e150 con 0x557740371860 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a78004230 con 0x557740371860 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 10 client.25561 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55774038c260 con 0x557740371860 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55774027e150 con 0x557740371860 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6a780052e0 con 0x557740371860 2024-08-20T06:41:55.423 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 10 client.25561 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 20 client.25561 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55774038d6e0 0x55774038fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 4 client.25561 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.421+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577402e1880 con 0x55774038d6e0 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.422+0000 7f6a90efa640 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55774038d6e0 0x55774038fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.422+0000 7f6a90efa640 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55774038d6e0 0x55774038fb00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b69685c1ce0407b server_cookie=f5293c9277ee4775 in_seq=0 out_seq=0 2024-08-20T06:41:55.424 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.422+0000 7f6a6bfff640 10 client.25561 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3427041285 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577402e1880 con 0x55774038d6e0 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a6bfff640 10 client.25561 handle_command_reply: tid=0 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 client.25561 shutdown 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 msgr2=0x55774037adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005ab0 tx=0x7f6a7800eb90 comp rx=0 tx=0).stop 2024-08-20T06:41:55.425 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 shutdown_connections 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55774037d7f0 0x55774037b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55774038d6e0 0x55774038fb00 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a6bfff640 0 client.25561 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740371860 0x55774037adc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 --2- 172.21.15.107:0/3427041285 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740370f40 0x55774037a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 >> 172.21.15.107:0/3427041285 conn(0x55774036ed70 msgr2=0x557740374910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 shutdown_connections 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 1 -- 172.21.15.107:0/3427041285 wait complete. 2024-08-20T06:41:55.426 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.424+0000 7f6a95557740 20 client.25561 trim_cache size 0 max 16384 2024-08-20T06:41:55.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 Processor -- start 2024-08-20T06:41:55.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 -- start start 2024-08-20T06:41:55.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740370f40 0x55774037a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.445 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 0x557740395420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1ae880 con 0x557740371860 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1ac150 con 0x557740370f40 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a95557740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1a9f80 con 0x55774037d7f0 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.443+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58416/0 (socket says 172.21.15.107:58416) 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 -- 172.21.15.107:0/1754268762 learned_addr learned my addr 172.21.15.107:0/1754268762 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a916fb640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 0x557740395420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a90efa640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740370f40 0x55774037a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 -- 172.21.15.107:0/1754268762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 msgr2=0x557740395420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.446 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 0x557740395420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 -- 172.21.15.107:0/1754268762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740370f40 msgr2=0x55774037a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740370f40 0x55774037a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 -- 172.21.15.107:0/1754268762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55774036eba0 con 0x557740371860 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a916fb640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 0x557740395420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6a7802eb60 tx=0x7f6a7800f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3dfd2a8e79b36b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.444+0000 7f6a767fc640 1 -- 172.21.15.107:0/1754268762 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7805b070 con 0x557740371860 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a767fc640 1 -- 172.21.15.107:0/1754268762 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a7800f990 con 0x557740371860 2024-08-20T06:41:55.447 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a767fc640 1 -- 172.21.15.107:0/1754268762 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78051080 con 0x557740371860 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 -- 172.21.15.107:0/1754268762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 msgr2=0x55774037af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6a7802eb60 tx=0x7f6a7800f660 comp rx=0 tx=0).stop 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 -- 172.21.15.107:0/1754268762 shutdown_connections 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55774037d7f0 0x557740395420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x557740371860 0x55774037af60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 --2- 172.21.15.107:0/1754268762 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740370f40 0x55774037a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 -- 172.21.15.107:0/1754268762 >> 172.21.15.107:0/1754268762 conn(0x55774036ed70 msgr2=0x55774037a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 -- 172.21.15.107:0/1754268762 shutdown_connections 2024-08-20T06:41:55.449 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.445+0000 7f6a95557740 1 -- 172.21.15.107:0/1754268762 wait complete. 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 -- start start 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 Processor -- start 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x5577403815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 0x557740382040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.446+0000 7f6a95557740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1aa610 con 0x55774037d7f0 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a95557740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1aba80 con 0x557740371860 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a95557740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f6a8c1aace0 con 0x557740370f40 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a90efa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x5577403815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a8bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44720/0 (socket says 172.21.15.107:44720) 2024-08-20T06:41:55.450 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a8bfff640 1 -- 172.21.15.107:0/3948051640 learned_addr learned my addr 172.21.15.107:0/3948051640 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 0x557740382040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a95557740 10 client.0 _ll_register_callbacks cb 0x7f6a80005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 -- 172.21.15.107:0/3948051640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 msgr2=0x5577403815c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x5577403815c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 -- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 msgr2=0x557740381b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a916fb640 1 -- 172.21.15.107:0/3948051640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557740365ce0 con 0x55774037d7f0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.447+0000 7f6a8bfff640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a916fb640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 0x557740382040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400fc70 tx=0x7f6a84013510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d6bb4cf1ee5990 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a6bfff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84013370 con 0x55774037d7f0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f6a8400ec10 con 0x55774037d7f0 2024-08-20T06:41:55.451 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a7800f990 con 0x55774037d7f0 2024-08-20T06:41:55.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a8400ee10 con 0x55774037d7f0 2024-08-20T06:41:55.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a95557740 10 client.16158 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:55.452 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577403f8610 con 0x55774037d7f0 2024-08-20T06:41:55.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.448+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f6a7800f990 con 0x55774037d7f0 2024-08-20T06:41:55.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6a84020050 con 0x55774037d7f0 2024-08-20T06:41:55.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 10 client.16158 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:55.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:55.453 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 20 client.16158 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5577403f8c70 0x5577403fb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 4 client.16158 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577402dd320 con 0x5577403f8c70 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.449+0000 7f6a90efa640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5577403f8c70 0x5577403fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.450+0000 7f6a6bfff640 10 client.16158 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.450+0000 7f6a90efa640 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5577403f8c70 0x5577403fb090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50df526727b9e053 server_cookie=ce91364884c42ed5 in_seq=0 out_seq=0 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.450+0000 7f6a6bfff640 1 -- 172.21.15.107:0/3948051640 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577402dd320 con 0x5577403f8c70 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.450+0000 7f6a6bfff640 10 client.16158 handle_command_reply: tid=0 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.450+0000 7f6a95557740 1 client.16158 shutdown 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 msgr2=0x557740382040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 0x557740382040 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6a8400fc70 tx=0x7f6a84013510 comp rx=0 tx=0).stop 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 shutdown_connections 2024-08-20T06:41:55.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55774037d7f0 0x557740382040 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x557740371860 0x557740381b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x5577403f8c70 0x5577403fb090 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 --2- 172.21.15.107:0/3948051640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557740370f40 0x5577403815c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 >> 172.21.15.107:0/3948051640 conn(0x55774036ed70 msgr2=0x5577403654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a6bfff640 0 client.16158 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 shutdown_connections 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 1 -- 172.21.15.107:0/3948051640 wait complete. 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.451+0000 7f6a95557740 20 client.16158 trim_cache size 0 max 16384 2024-08-20T06:41:55.455 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:41:55.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 msgr2=0x7f6a7c0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 0x7f6a7c0b61d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6a84005fd0 tx=0x7f6a84009b70 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 msgr2=0x7f6a8c196ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000e9f0 tx=0x7f6a8000eeb0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 shutdown_connections 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6a8c10ce00 0x7f6a8c19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f6a7c0b3d10 0x7f6a7c0b61d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a8c1042c0 0x7f6a8c197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 --2- 172.21.15.107:0/946164269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f6a8c1039a0 0x7f6a8c196ad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.452+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 >> 172.21.15.107:0/946164269 conn(0x7f6a8c0fd400 msgr2=0x7f6a8c10efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.453+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 shutdown_connections 2024-08-20T06:41:55.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:55.453+0000 7f6a9315e640 1 -- 172.21.15.107:0/946164269 wait complete. 2024-08-20T06:41:55.887 INFO:tasks.workunit.client.0.smithi107.stdout: 78 12409 164 8061 66 5099 343 2024-08-20T06:41:56.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:56 smithi183 ceph-mon[22204]: pgmap v480: 97 pgs: 97 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-08-20T06:41:56.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:56 smithi193 ceph-mon[21944]: pgmap v480: 97 pgs: 97 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-08-20T06:41:56.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:56 smithi107 ceph-mon[18495]: pgmap v480: 97 pgs: 97 active+clean; 356 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 292 op/s 2024-08-20T06:41:57.196 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:41:57.196 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:41:57.551 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.179+0000 7f5d97f9e640 1 -- 172.21.15.107:0/3088616305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90108e20 msgr2=0x7f5d9010f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.179+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/3088616305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90108e20 0x7f5d9010f620 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5d88009ca0 tx=0x7f5d8802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 -- 172.21.15.107:0/3088616305 shutdown_connections 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/3088616305 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90108e20 0x7f5d9010f620 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/3088616305 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d901084a0 0x7f5d901088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/3088616305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d90102a90 0x7f5d90102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.184 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 -- 172.21.15.107:0/3088616305 >> 172.21.15.107:0/3088616305 conn(0x7f5d900fe540 msgr2=0x7f5d90100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 -- 172.21.15.107:0/3088616305 shutdown_connections 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 -- 172.21.15.107:0/3088616305 wait complete. 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 Processor -- start 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.180+0000 7f5d97f9e640 1 -- start start 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d901084a0 0x7f5d9010ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 0x7f5d901a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58444/0 (socket says 172.21.15.107:58444) 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2167975055 learned_addr learned my addr 172.21.15.107:0/2167975055 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901129e0 con 0x7f5d90102a90 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90112860 con 0x7f5d90108e20 2024-08-20T06:41:58.185 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d90112b60 con 0x7f5d901084a0 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 0x7f5d901a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d95539640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d901084a0 0x7f5d9010ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 -- 172.21.15.107:0/2167975055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d901084a0 msgr2=0x7f5d9010ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d901084a0 0x7f5d9010ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 -- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 msgr2=0x7f5d9010e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d9653b640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d901a5580 con 0x7f5d90108e20 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.181+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 0x7f5d901a4db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031040 tx=0x7f5d88003f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d86ffd640 1 -- 172.21.15.107:0/2167975055 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88032c90 con 0x7f5d90108e20 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d901a5810 con 0x7f5d90108e20 2024-08-20T06:41:58.186 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d86ffd640 1 -- 172.21.15.107:0/2167975055 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d88032e30 con 0x7f5d90108e20 2024-08-20T06:41:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d86ffd640 1 -- 172.21.15.107:0/2167975055 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88030b20 con 0x7f5d90108e20 2024-08-20T06:41:58.187 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.182+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d901abf30 con 0x7f5d90108e20 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.185+0000 7f5d9a397740 1 Processor -- start 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.185+0000 7f5d86ffd640 1 -- 172.21.15.107:0/2167975055 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f5d88030cc0 con 0x7f5d90108e20 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.185+0000 7f5d9a397740 1 -- start start 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 0x55d58d859790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d868050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d58d7cf940 con 0x55d58d8593b0 2024-08-20T06:41:58.189 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d58d7bee40 con 0x55d58d865c60 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d9a397740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d58d7c2e80 con 0x55d58d859cd0 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95539640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 0x55d58d859790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:57610/0 (socket says 172.21.15.107:57610) 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 -- 172.21.15.107:0/4292342747 learned_addr learned my addr 172.21.15.107:0/4292342747 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95539640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 0x55d58d859790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58452/0 (socket says 172.21.15.107:58452) 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95539640 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d868050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 -- 172.21.15.107:0/4292342747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 msgr2=0x55d58d868050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d868050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 -- 172.21.15.107:0/4292342747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 msgr2=0x55d58d859790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 0x55d58d859790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.190 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 -- 172.21.15.107:0/4292342747 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58d75b520 con 0x55d58d859cd0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.186+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c010590 tx=0x7f5d8c010a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef961c2b91d22ef7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d7bfff640 1 -- 172.21.15.107:0/4292342747 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00e770 con 0x55d58d859cd0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d7bfff640 1 -- 172.21.15.107:0/4292342747 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d8c00eda0 con 0x55d58d859cd0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d7bfff640 1 -- 172.21.15.107:0/4292342747 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c011d10 con 0x55d58d859cd0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d86ffd640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 0x7f5d7c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d95539640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 0x7f5d7c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.187+0000 7f5d86ffd640 1 -- 172.21.15.107:0/2167975055 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d88104f20 con 0x7f5d90108e20 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 -- 172.21.15.107:0/4292342747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 msgr2=0x55d58d865720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c010590 tx=0x7f5d8c010a50 comp rx=0 tx=0).stop 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d95539640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 0x7f5d7c0b6110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d58d859360 tx=0x7f5d800096c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 -- 172.21.15.107:0/4292342747 shutdown_connections 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d868050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.191 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d859cd0 0x55d58d865720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 --2- 172.21.15.107:0/4292342747 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d8593b0 0x55d58d859790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 -- 172.21.15.107:0/4292342747 >> 172.21.15.107:0/4292342747 conn(0x55d58d8571e0 msgr2=0x55d58d858380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 -- 172.21.15.107:0/4292342747 shutdown_connections 2024-08-20T06:41:58.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.188+0000 7f5d9a397740 1 -- 172.21.15.107:0/4292342747 wait complete. 2024-08-20T06:41:58.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.189+0000 7f5d9a397740 1 -- start start 2024-08-20T06:41:58.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.189+0000 7f5d9a397740 1 Processor -- start 2024-08-20T06:41:58.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d8630c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 0x55d58d863600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55d58d621890 con 0x55d58d859cd0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44758/0 (socket says 172.21.15.107:44758) 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2162066030 learned_addr learned my addr 172.21.15.107:0/2162066030 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d95539640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d8630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55d58d745700 con 0x55d58d8593b0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 0x55d58d863600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55d58d8689b0 con 0x55d58d865c60 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.190+0000 7f5d9a397740 10 client.0 _ll_register_callbacks cb 0x55d58d7ba6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:58.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2162066030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 msgr2=0x55d58d863600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 0x55d58d863600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2162066030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 msgr2=0x55d58d8630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d8630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58d51e930 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 0x55d58d863600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005760 tx=0x7f5d8c011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae36af8e57d23de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d797fa640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c027020 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d8c011440 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c00eda0 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c01f040 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d9a397740 10 client.25685 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.191+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d58d8744d0 con 0x55d58d8593b0 2024-08-20T06:41:58.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d8c00eda0 con 0x55d58d8593b0 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5d8c022030 con 0x55d58d8593b0 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 10 client.25685 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 20 client.25685 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d877c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 4 client.25685 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d58d7d1890 con 0x55d58d875880 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.192+0000 7f5d95539640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d877c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.193+0000 7f5d95539640 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d877c70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7a1256deae9e9e server_cookie=1c71da707bb114d9 in_seq=0 out_seq=0 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.193+0000 7f5d797fa640 10 client.25685 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d797fa640 1 -- 172.21.15.107:0/2162066030 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d58d7d1890 con 0x55d58d875880 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d797fa640 10 client.25685 handle_command_reply: tid=0 2024-08-20T06:41:58.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 client.25685 shutdown 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 msgr2=0x55d58d862b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c005760 tx=0x7f5d8c011a90 comp rx=0 tx=0).stop 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 shutdown_connections 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d877c70 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d797fa640 0 client.25685 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d865c60 0x55d58d863600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d8630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2162066030 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d8593b0 0x55d58d862b80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 >> 172.21.15.107:0/2162066030 conn(0x55d58d8571e0 msgr2=0x55d58d85cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 shutdown_connections 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 1 -- 172.21.15.107:0/2162066030 wait complete. 2024-08-20T06:41:58.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.194+0000 7f5d9a397740 20 client.25685 trim_cache size 0 max 16384 2024-08-20T06:41:58.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 Processor -- start 2024-08-20T06:41:58.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 -- start start 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 0x55d58d871200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d865c60 0x55d58d871740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b6f00 con 0x55d58d865c60 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b47d0 con 0x55d58d859cd0 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.221+0000 7f5d9a397740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b2600 con 0x55d58d8593b0 2024-08-20T06:41:58.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95539640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 0x55d58d871200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95539640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 0x55d58d871200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44762/0 (socket says 172.21.15.107:44762) 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:57632/0 (socket says 172.21.15.107:57632) 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95539640 1 -- 172.21.15.107:0/2526354577 learned_addr learned my addr 172.21.15.107:0/2526354577 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d9653b640 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d865c60 0x55d58d871740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2526354577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 msgr2=0x55d58d871200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 0x55d58d871200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2526354577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d865c60 msgr2=0x55d58d871740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d865c60 0x55d58d871740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 -- 172.21.15.107:0/2526354577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d58d858910 con 0x55d58d8593b0 2024-08-20T06:41:58.224 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.222+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00ef90 tx=0x7f5d8c010f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a0c927bf699f3d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.223+0000 7f5d7b7fe640 1 -- 172.21.15.107:0/2526354577 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c027040 con 0x55d58d8593b0 2024-08-20T06:41:58.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.223+0000 7f5d7b7fe640 1 -- 172.21.15.107:0/2526354577 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f5d8c011640 con 0x55d58d8593b0 2024-08-20T06:41:58.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.223+0000 7f5d7b7fe640 1 -- 172.21.15.107:0/2526354577 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c01ae50 con 0x55d58d8593b0 2024-08-20T06:41:58.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 -- 172.21.15.107:0/2526354577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 msgr2=0x55d58d85d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.226 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00ef90 tx=0x7f5d8c010f50 comp rx=0 tx=0).stop 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 -- 172.21.15.107:0/2526354577 shutdown_connections 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d865c60 0x55d58d871740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d859cd0 0x55d58d871200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 --2- 172.21.15.107:0/2526354577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d85d4b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 -- 172.21.15.107:0/2526354577 >> 172.21.15.107:0/2526354577 conn(0x55d58d8571e0 msgr2=0x55d58d870af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 -- 172.21.15.107:0/2526354577 shutdown_connections 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.224+0000 7f5d9a397740 1 -- 172.21.15.107:0/2526354577 wait complete. 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.225+0000 7f5d9a397740 1 -- start start 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.225+0000 7f5d9a397740 1 Processor -- start 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.225+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.227 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.225+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d86a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d8df3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b2c90 con 0x55d58d859cd0 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b4100 con 0x55d58d865c60 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f5d901b3360 con 0x55d58d8593b0 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:57638/0 (socket says 172.21.15.107:57638) 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95539640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d86a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 -- 172.21.15.107:0/543609773 learned_addr learned my addr 172.21.15.107:0/543609773 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9653b640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d8df3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.229 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 -- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 msgr2=0x55d58d8df3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d8df3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 -- 172.21.15.107:0/543609773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 msgr2=0x55d58d86a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d86a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 -- 172.21.15.107:0/543609773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d58d8dfa70 con 0x55d58d8593b0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 10 client.0 _ll_register_callbacks cb 0x7f5d800099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d8c011640 con 0x55d58d8593b0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d95d3a640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00bdc0 tx=0x7f5d8c0056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8834798e3961244d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d797fa640 10 client.25585 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d797fa640 1 -- 172.21.15.107:0/543609773 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c046050 con 0x55d58d8593b0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.226+0000 7f5d797fa640 1 -- 172.21.15.107:0/543609773 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f5d8c011440 con 0x55d58d8593b0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d797fa640 1 -- 172.21.15.107:0/543609773 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5d8c03b030 con 0x55d58d8593b0 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 10 client.25585 fetch_fsmap learned FSMap version 29 2024-08-20T06:41:58.230 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55d58d8e0be0 con 0x55d58d8593b0 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d797fa640 1 -- 172.21.15.107:0/543609773 <== mon.1 v2:172.21.15.193:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5d8c049020 con 0x55d58d8593b0 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 10 client.25585 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 10 client.25585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 20 client.25585 populate_metadata read hostname 'smithi107' 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d8e3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 4 client.25585 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d58d7cd330 con 0x55d58d875880 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.227+0000 7f5d95539640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d8e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.228+0000 7f5d95539640 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d8e3310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=345ecfbf27b7833a server_cookie=378b4acbd58abf3e in_seq=0 out_seq=0 2024-08-20T06:41:58.231 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.228+0000 7f5d797fa640 10 client.25585 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:58.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d797fa640 1 -- 172.21.15.107:0/543609773 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d58d7cd330 con 0x55d58d875880 2024-08-20T06:41:58.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d797fa640 10 client.25585 handle_command_reply: tid=0 2024-08-20T06:41:58.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 client.25585 shutdown 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 msgr2=0x55d58d86a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00bdc0 tx=0x7f5d8c0056f0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 shutdown_connections 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55d58d865c60 0x55d58d8df3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55d58d859cd0 0x55d58d86a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55d58d875880 0x55d58d8e3310 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d797fa640 0 client.25585 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 --2- 172.21.15.107:0/543609773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d58d8593b0 0x55d58d86a2f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 >> 172.21.15.107:0/543609773 conn(0x55d58d8571e0 msgr2=0x55d58d871310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 shutdown_connections 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 1 -- 172.21.15.107:0/543609773 wait complete. 2024-08-20T06:41:58.233 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.229+0000 7f5d9a397740 20 client.25585 trim_cache size 0 max 16384 2024-08-20T06:41:58.234 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:41:58.234 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:41:58.234 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "7dad823e-747f-49f3-8eea-152e81beeb57", 2024-08-20T06:41:58.234 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:41:58.234 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 msgr2=0x7f5d7c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 0x7f5d7c0b6110 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55d58d859360 tx=0x7f5d800096c0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 msgr2=0x7f5d901a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 0x7f5d901a4db0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d88031040 tx=0x7f5d88003f80 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 shutdown_connections 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f5d90108e20 0x7f5d901a4db0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f5d7c0b3c50 0x7f5d7c0b6110 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5d901084a0 0x7f5d9010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 --2- 172.21.15.107:0/2167975055 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5d90102a90 0x7f5d9010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:41:58.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 >> 172.21.15.107:0/2167975055 conn(0x7f5d900fe540 msgr2=0x7f5d90104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:41:58.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 shutdown_connections 2024-08-20T06:41:58.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:41:58.230+0000 7f5d97f9e640 1 -- 172.21.15.107:0/2167975055 wait complete. 2024-08-20T06:41:58.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:58 smithi107 ceph-mon[18495]: pgmap v481: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-08-20T06:41:58.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:58 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:41:58.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:58 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:58.727 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:58 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:58 smithi183 ceph-mon[22204]: pgmap v481: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-08-20T06:41:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:58 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:41:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:58 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:58.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:58 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:58 smithi193 ceph-mon[21944]: pgmap v481: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 311 op/s 2024-08-20T06:41:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:58 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:41:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:58 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:58.815 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:58 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.168 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:41:59.555 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:41:59.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:41:59 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:41:59.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:41:59 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:41:59.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:41:59 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:42:00.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:00 smithi183 ceph-mon[22204]: pgmap v482: 97 pgs: 97 active+clean; 363 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-08-20T06:42:00.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:00 smithi193 ceph-mon[21944]: pgmap v482: 97 pgs: 97 active+clean; 363 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-08-20T06:42:00.819 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:00 smithi107 ceph-mon[18495]: pgmap v482: 97 pgs: 97 active+clean; 363 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-08-20T06:42:01.735 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:01 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:42:01.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:01 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:42:01.824 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:01 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:42:02.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:02 smithi183 ceph-mon[22204]: pgmap v483: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 443 op/s 2024-08-20T06:42:02.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:02 smithi193 ceph-mon[21944]: pgmap v483: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 443 op/s 2024-08-20T06:42:02.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:02 smithi107 ceph-mon[18495]: pgmap v483: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 443 op/s 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 -- 172.21.15.107:0/1231327599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f0101990 msgr2=0x7f20f0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 --2- 172.21.15.107:0/1231327599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f0101990 0x7f20f0101d70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f20ec009860 tx=0x7f20ec02a340 comp rx=0 tx=0).stop 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 -- 172.21.15.107:0/1231327599 shutdown_connections 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 --2- 172.21.15.107:0/1231327599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0107d20 0x7f20f010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 --2- 172.21.15.107:0/1231327599 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f01073a0 0x7f20f01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 --2- 172.21.15.107:0/1231327599 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f0101990 0x7f20f0101d70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.935 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 -- 172.21.15.107:0/1231327599 >> 172.21.15.107:0/1231327599 conn(0x7f20f00fd400 msgr2=0x7f20f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.037+0000 7f20f7701640 1 -- 172.21.15.107:0/1231327599 shutdown_connections 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.038+0000 7f20f7701640 1 -- 172.21.15.107:0/1231327599 wait complete. 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.038+0000 7f20f7701640 1 Processor -- start 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.038+0000 7f20f7701640 1 -- start start 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0101990 0x7f20f019b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f0107d20 0x7f20f01a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114b90 con 0x7f20f0107d20 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114a10 con 0x7f20f01073a0 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f7701640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114d10 con 0x7f20f0101990 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.936 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44790/0 (socket says 172.21.15.107:44790) 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 -- 172.21.15.107:0/3562291049 learned_addr learned my addr 172.21.15.107:0/3562291049 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f0107d20 0x7f20f01a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f549d640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0101990 0x7f20f019b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 -- 172.21.15.107:0/3562291049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0101990 msgr2=0x7f20f019b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0101990 0x7f20f019b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 -- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f0107d20 msgr2=0x7f20f01a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f0107d20 0x7f20f01a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 -- 172.21.15.107:0/3562291049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f01a1150 con 0x7f20f01073a0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.039+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20e000e9c0 tx=0x7f20e000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.040+0000 7f20e67fc640 1 -- 172.21.15.107:0/3562291049 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e000cde0 con 0x7f20f01073a0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.040+0000 7f20e67fc640 1 -- 172.21.15.107:0/3562291049 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20e00035f0 con 0x7f20f01073a0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.040+0000 7f20e67fc640 1 -- 172.21.15.107:0/3562291049 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0010690 con 0x7f20f01073a0 2024-08-20T06:42:03.937 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.040+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01a1440 con 0x7f20f01073a0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.040+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a7b00 con 0x7f20f01073a0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.043+0000 7f20f9afa740 1 Processor -- start 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.043+0000 7f20f9afa740 1 -- start start 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 0x556989dc0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dc07a0 0x556989dc2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556989d228e0 con 0x556989dac000 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556989d11de0 con 0x556989dab6e0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f9afa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556989d15e20 con 0x556989dc07a0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20e67fc640 1 -- 172.21.15.107:0/3562291049 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f20e0004000 con 0x7f20f01073a0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.938 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f549d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 0x556989dc0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44800/0 (socket says 172.21.15.107:44800) 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 -- 172.21.15.107:0/1386739245 learned_addr learned my addr 172.21.15.107:0/1386739245 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dc07a0 0x556989dc2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 -- 172.21.15.107:0/1386739245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dc07a0 msgr2=0x556989dc2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dc07a0 0x556989dc2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 -- 172.21.15.107:0/1386739245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 msgr2=0x556989dc0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 0x556989dc0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f5c9e640 1 -- 172.21.15.107:0/1386739245 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989a71650 con 0x556989dab6e0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.044+0000 7f20f549d640 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 0x556989dc0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.045+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20e8010370 tx=0x7f20e8010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0a64ed00a9e7c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.045+0000 7f20db7fe640 1 -- 172.21.15.107:0/1386739245 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8016070 con 0x556989dab6e0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.045+0000 7f20db7fe640 1 -- 172.21.15.107:0/1386739245 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20e8004d20 con 0x556989dab6e0 2024-08-20T06:42:03.939 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.045+0000 7f20db7fe640 1 -- 172.21.15.107:0/1386739245 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e80052c0 con 0x556989dab6e0 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.045+0000 7f20e67fc640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 0x7f20dc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f549d640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 0x7f20dc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20e67fc640 1 -- 172.21.15.107:0/3562291049 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f20e00e3e80 con 0x7f20f01073a0 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 -- 172.21.15.107:0/1386739245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 msgr2=0x556989dabac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20e8010370 tx=0x7f20e8010830 comp rx=0 tx=0).stop 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 -- 172.21.15.107:0/1386739245 shutdown_connections 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dc07a0 0x556989dc2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dac000 0x556989dc0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1386739245 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dab6e0 0x556989dabac0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 -- 172.21.15.107:0/1386739245 >> 172.21.15.107:0/1386739245 conn(0x556989da9510 msgr2=0x556989daa6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 -- 172.21.15.107:0/1386739245 shutdown_connections 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f549d640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 0x7f20dc0b6170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20ec02a930 tx=0x7f20ec005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.940 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.046+0000 7f20f9afa740 1 -- 172.21.15.107:0/1386739245 wait complete. 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 -- start start 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 Processor -- start 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dbd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dbdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556989b74700 con 0x556989dc07a0 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556989c987c0 con 0x556989dac000 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556989dc3420 con 0x556989dab6e0 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f9afa740 10 client.0 _ll_register_callbacks cb 0x556989d0b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:03.941 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "IDLE (waiting for more scrubs)", 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "7dad823e-747f-49f3-8eea-152e81beeb57": { 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "7dad823e-747f-49f3-8eea-152e81beeb57", 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:42:03.942 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dbdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.047+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dbdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44810/0 (socket says 172.21.15.107:44810) 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f4c9c640 1 -- 172.21.15.107:0/3141057554 learned_addr learned my addr 172.21.15.107:0/3141057554 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f549d640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dbd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 -- 172.21.15.107:0/3141057554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 msgr2=0x556989dbd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dbd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 -- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 msgr2=0x556989dbdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dbdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 -- 172.21.15.107:0/3141057554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556989dcc880 con 0x556989dc07a0 2024-08-20T06:42:03.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f5c9e640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dbe270 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20e8019000 tx=0x7f20e8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185f1b25b662a5a3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8027020 con 0x556989dc07a0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f20e8004240 con 0x556989dc07a0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556989cc1100 con 0x556989dc07a0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20e8016040 con 0x556989dc07a0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f9afa740 10 client.16188 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.048+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556989dcf230 con 0x556989dc07a0 2024-08-20T06:42:03.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556989cc1100 con 0x556989dc07a0 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f20e8022070 con 0x556989dc07a0 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 10 client.16188 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 10 client.16188 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 20 client.16188 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989dd0990 0x556989dd2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 4 client.16188 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556989d24830 con 0x556989dd0990 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.049+0000 7f20f549d640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989dd0990 0x556989dd2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.050+0000 7f20d8ff9640 10 client.16188 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.050+0000 7f20f549d640 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989dd0990 0x556989dd2dd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f03760ffc6a8d28a server_cookie=6a0919c84ed480d2 in_seq=0 out_seq=0 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.050+0000 7f20d8ff9640 1 -- 172.21.15.107:0/3141057554 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556989d24830 con 0x556989dd0990 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.050+0000 7f20d8ff9640 10 client.16188 handle_command_reply: tid=0 2024-08-20T06:42:03.945 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.050+0000 7f20f9afa740 1 client.16188 shutdown 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 msgr2=0x556989dbe270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dbe270 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20e8019000 tx=0x7f20e8003ba0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 shutdown_connections 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dbe270 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dbdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989dd0990 0x556989dd2dd0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 --2- 172.21.15.107:0/3141057554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dbd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 >> 172.21.15.107:0/3141057554 conn(0x556989da9510 msgr2=0x556989daf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20d8ff9640 0 client.16188 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 shutdown_connections 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 1 -- 172.21.15.107:0/3141057554 wait complete. 2024-08-20T06:42:03.946 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.051+0000 7f20f9afa740 20 client.16188 trim_cache size 0 max 16384 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 Processor -- start 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 -- start start 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 0x556989daad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 0x556989dbb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01ae6b0 con 0x556989dab6e0 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01abf80 con 0x556989dc07a0 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.075+0000 7f20f9afa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01a9db0 con 0x556989dac000 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f549d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 0x556989daad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 0x556989dbb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:57680/0 (socket says 172.21.15.107:57680) 2024-08-20T06:42:03.947 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f549d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 0x556989daad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58534/0 (socket says 172.21.15.107:58534) 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 -- 172.21.15.107:0/1170031712 learned_addr learned my addr 172.21.15.107:0/1170031712 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 0x556989dbb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44822/0 (socket says 172.21.15.107:44822) 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 -- 172.21.15.107:0/1170031712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 msgr2=0x556989dbb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 0x556989dbb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 -- 172.21.15.107:0/1170031712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 msgr2=0x556989daad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 0x556989daad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 -- 172.21.15.107:0/1170031712 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556989dbc750 con 0x556989dac000 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e00e7380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=491d47981e1acb88 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20daffd640 1 -- 172.21.15.107:0/1170031712 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00aa500 con 0x556989dac000 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20daffd640 1 -- 172.21.15.107:0/1170031712 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20e00aa8a0 con 0x556989dac000 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.076+0000 7f20daffd640 1 -- 172.21.15.107:0/1170031712 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00042f0 con 0x556989dac000 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 -- 172.21.15.107:0/1170031712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 msgr2=0x556989dbcce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.948 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f20e0000c00 tx=0x7f20e00e7380 comp rx=0 tx=0).stop 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 -- 172.21.15.107:0/1170031712 shutdown_connections 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dc07a0 0x556989dbb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dac000 0x556989dbcce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 --2- 172.21.15.107:0/1170031712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dab6e0 0x556989daad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 -- 172.21.15.107:0/1170031712 >> 172.21.15.107:0/1170031712 conn(0x556989da9510 msgr2=0x556989daa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 -- 172.21.15.107:0/1170031712 shutdown_connections 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.077+0000 7f20f9afa740 1 -- 172.21.15.107:0/1170031712 wait complete. 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 -- start start 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 Processor -- start 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dd8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dc8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.949 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01aa440 con 0x556989dc07a0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01ab8b0 con 0x556989dac000 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01aab10 con 0x556989dab6e0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.078+0000 7f20f9afa740 10 client.0 _ll_register_callbacks cb 0x7f20e0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:44826/0 (socket says 172.21.15.107:44826) 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f5c9e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dc8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 -- 172.21.15.107:0/361020145 learned_addr learned my addr 172.21.15.107:0/361020145 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f549d640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dd8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 -- 172.21.15.107:0/361020145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 msgr2=0x556989dd8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dd8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 -- 172.21.15.107:0/361020145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 msgr2=0x556989dc8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dc8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.950 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 -- 172.21.15.107:0/361020145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556989dabe30 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f549d640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dd8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f4c9c640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009230 tx=0x7f20e0003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebebd9d4e2a8bab4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e00c4020 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f20e00a7380 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.079+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e00aa8a0 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f20e00aa500 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 10 client.25709 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556989e3a710 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f20e00aa8a0 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f20e00b39b0 con 0x556989dac000 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 10 client.25709 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 10 client.25709 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 20 client.25709 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:03.951 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989e3ad90 0x556989e3d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 4 client.25709 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.080+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556989d202d0 con 0x556989e3ad90 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.081+0000 7f20f549d640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989e3ad90 0x556989e3d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.081+0000 7f20f549d640 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989e3ad90 0x556989e3d1b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89c75aecd750a15 server_cookie=eeeb662a85570efe in_seq=0 out_seq=0 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.081+0000 7f20d8ff9640 10 client.25709 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.083+0000 7f20d8ff9640 1 -- 172.21.15.107:0/361020145 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x556989d202d0 con 0x556989e3ad90 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.083+0000 7f20d8ff9640 10 client.25709 handle_command_reply: tid=0 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.083+0000 7f20f9afa740 1 client.25709 shutdown 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.083+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 msgr2=0x556989dc86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009230 tx=0x7f20e0003cd0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 shutdown_connections 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556989dc07a0 0x556989dc8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556989dac000 0x556989dc86b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.952 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556989e3ad90 0x556989e3d1b0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 --2- 172.21.15.107:0/361020145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556989dab6e0 0x556989dd8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 >> 172.21.15.107:0/361020145 conn(0x556989da9510 msgr2=0x556989da7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20d8ff9640 0 client.25709 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 shutdown_connections 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 1 -- 172.21.15.107:0/361020145 wait complete. 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.084+0000 7f20f9afa740 20 client.25709 trim_cache size 0 max 16384 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 msgr2=0x7f20dc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 0x7f20dc0b6170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20ec02a930 tx=0x7f20ec005d30 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 msgr2=0x7f20f019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f20e000e9c0 tx=0x7f20e000ee80 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 shutdown_connections 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f20f0107d20 0x7f20f01a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f20f01073a0 0x7f20f019b5e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f20dc0b3cb0 0x7f20dc0b6170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 --2- 172.21.15.107:0/3562291049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f0101990 0x7f20f019b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 >> 172.21.15.107:0/3562291049 conn(0x7f20f00fd400 msgr2=0x7f20f00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 shutdown_connections 2024-08-20T06:42:03.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:03.085+0000 7f20f7701640 1 -- 172.21.15.107:0/3562291049 wait complete. 2024-08-20T06:42:04.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:04 smithi183 ceph-mon[22204]: pgmap v484: 97 pgs: 97 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 856 op/s 2024-08-20T06:42:04.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:04 smithi193 ceph-mon[21944]: pgmap v484: 97 pgs: 97 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 856 op/s 2024-08-20T06:42:04.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:04 smithi107 ceph-mon[18495]: pgmap v484: 97 pgs: 97 active+clean; 359 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.0 MiB/s wr, 856 op/s 2024-08-20T06:42:05.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:05 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:42:05.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:05 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:42:05.886 INFO:tasks.workunit.client.0.smithi107.stdout:read(): Bad file descriptor 79 11061 134 7030 41 4821 282 2024-08-20T06:42:05.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:05 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:42:06.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:06 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:42:06.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:06 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:42:06.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:06 smithi193 ceph-mon[21944]: pgmap v485: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 849 op/s 2024-08-20T06:42:06.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:06 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:42:06.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:06 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:42:06.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:06 smithi107 ceph-mon[18495]: pgmap v485: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 849 op/s 2024-08-20T06:42:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:06 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:42:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:06 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:42:06.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:06 smithi183 ceph-mon[22204]: pgmap v485: 97 pgs: 97 active+clean; 360 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 849 op/s 2024-08-20T06:42:08.806 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:08 smithi193 ceph-mon[21944]: pgmap v486: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 866 op/s 2024-08-20T06:42:08.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:08 smithi107 ceph-mon[18495]: pgmap v486: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 866 op/s 2024-08-20T06:42:08.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:08 smithi183 ceph-mon[22204]: pgmap v486: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 866 op/s 2024-08-20T06:42:10.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:10 smithi193 ceph-mon[21944]: pgmap v487: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 849 op/s 2024-08-20T06:42:10.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:10 smithi107 ceph-mon[18495]: pgmap v487: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 849 op/s 2024-08-20T06:42:10.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:10 smithi183 ceph-mon[22204]: pgmap v487: 97 pgs: 97 active+clean; 365 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 849 op/s 2024-08-20T06:42:11.292 INFO:tasks.cephfs.filesystem:scrub status for tag:7dad823e-747f-49f3-8eea-152e81beeb57 - {'path': '/', 'tag': '7dad823e-747f-49f3-8eea-152e81beeb57', 'options': 'recursive,force'} 2024-08-20T06:42:12.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:12 smithi193 ceph-mon[21944]: pgmap v488: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 850 op/s 2024-08-20T06:42:12.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:12 smithi107 ceph-mon[18495]: pgmap v488: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 850 op/s 2024-08-20T06:42:12.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:12 smithi183 ceph-mon[22204]: pgmap v488: 97 pgs: 97 active+clean; 362 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.5 MiB/s wr, 850 op/s 2024-08-20T06:42:13.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:42:13.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:42:13.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:42:13.805 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:13 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:42:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:42:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:42:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:42:13.861 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:13 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:42:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:42:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:42:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:42:13.986 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:13 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:42:15.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:15 smithi193 ceph-mon[21944]: pgmap v489: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 723 op/s 2024-08-20T06:42:15.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:15 smithi107 ceph-mon[18495]: pgmap v489: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 723 op/s 2024-08-20T06:42:15.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:15 smithi183 ceph-mon[22204]: pgmap v489: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 723 op/s 2024-08-20T06:42:15.887 INFO:tasks.workunit.client.0.smithi107.stdout: 79 12496 30 8088 150 3942 270 2024-08-20T06:42:17.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:17 smithi193 ceph-mon[21944]: pgmap v490: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-08-20T06:42:17.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:17 smithi107 ceph-mon[18495]: pgmap v490: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-08-20T06:42:17.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:17 smithi183 ceph-mon[22204]: pgmap v490: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 285 op/s 2024-08-20T06:42:19.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:19 smithi193 ceph-mon[21944]: pgmap v491: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T06:42:19.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:19 smithi107 ceph-mon[18495]: pgmap v491: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T06:42:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:19 smithi183 ceph-mon[22204]: pgmap v491: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T06:42:21.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:21 smithi193 ceph-mon[21944]: pgmap v492: 97 pgs: 97 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-08-20T06:42:21.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:21 smithi107 ceph-mon[18495]: pgmap v492: 97 pgs: 97 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-08-20T06:42:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:21 smithi183 ceph-mon[22204]: pgmap v492: 97 pgs: 97 active+clean; 378 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 248 op/s 2024-08-20T06:42:23.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:23 smithi193 ceph-mon[21944]: pgmap v493: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-08-20T06:42:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:23 smithi107 ceph-mon[18495]: pgmap v493: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-08-20T06:42:23.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:23 smithi183 ceph-mon[22204]: pgmap v493: 97 pgs: 97 active+clean; 372 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 245 op/s 2024-08-20T06:42:25.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:25 smithi107 ceph-mon[18495]: pgmap v494: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-08-20T06:42:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:25 smithi183 ceph-mon[22204]: pgmap v494: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-08-20T06:42:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:25 smithi193 ceph-mon[21944]: pgmap v494: 97 pgs: 97 active+clean; 371 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.3 MiB/s wr, 288 op/s 2024-08-20T06:42:25.887 INFO:tasks.workunit.client.0.smithi107.stdout: 81 13646 136 8257 123 5267 296 2024-08-20T06:42:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:27 smithi107 ceph-mon[18495]: pgmap v495: 97 pgs: 97 active+clean; 369 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-08-20T06:42:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:27 smithi183 ceph-mon[22204]: pgmap v495: 97 pgs: 97 active+clean; 369 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-08-20T06:42:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:27 smithi193 ceph-mon[21944]: pgmap v495: 97 pgs: 97 active+clean; 369 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 249 op/s 2024-08-20T06:42:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:29 smithi107 ceph-mon[18495]: pgmap v496: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-08-20T06:42:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:29 smithi183 ceph-mon[22204]: pgmap v496: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-08-20T06:42:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:29 smithi193 ceph-mon[21944]: pgmap v496: 97 pgs: 97 active+clean; 376 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 282 op/s 2024-08-20T06:42:31.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:31 smithi107 ceph-mon[18495]: pgmap v497: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-08-20T06:42:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:31 smithi183 ceph-mon[22204]: pgmap v497: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-08-20T06:42:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:31 smithi193 ceph-mon[21944]: pgmap v497: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 294 op/s 2024-08-20T06:42:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:33 smithi107 ceph-mon[18495]: pgmap v498: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2024-08-20T06:42:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:33 smithi183 ceph-mon[22204]: pgmap v498: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2024-08-20T06:42:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:33 smithi193 ceph-mon[21944]: pgmap v498: 97 pgs: 97 active+clean; 379 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 303 op/s 2024-08-20T06:42:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:35 smithi183 ceph-mon[22204]: pgmap v499: 97 pgs: 97 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-08-20T06:42:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:35 smithi193 ceph-mon[21944]: pgmap v499: 97 pgs: 97 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-08-20T06:42:35.887 INFO:tasks.workunit.client.0.smithi107.stdout: 84 13030 99 8566 156 5852 284 2024-08-20T06:42:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:35 smithi107 ceph-mon[18495]: pgmap v499: 97 pgs: 97 active+clean; 380 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 351 op/s 2024-08-20T06:42:37.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:37 smithi107 ceph-mon[18495]: pgmap v500: 97 pgs: 97 active+clean; 381 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-08-20T06:42:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:37 smithi183 ceph-mon[22204]: pgmap v500: 97 pgs: 97 active+clean; 381 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-08-20T06:42:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:37 smithi193 ceph-mon[21944]: pgmap v500: 97 pgs: 97 active+clean; 381 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 299 op/s 2024-08-20T06:42:39.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:39 smithi107 ceph-mon[18495]: pgmap v501: 97 pgs: 97 active+clean; 390 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2024-08-20T06:42:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:39 smithi183 ceph-mon[22204]: pgmap v501: 97 pgs: 97 active+clean; 390 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2024-08-20T06:42:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:39 smithi193 ceph-mon[21944]: pgmap v501: 97 pgs: 97 active+clean; 390 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 325 op/s 2024-08-20T06:42:41.294 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:42:41.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:41 smithi107 ceph-mon[18495]: pgmap v502: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-08-20T06:42:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:41 smithi183 ceph-mon[22204]: pgmap v502: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-08-20T06:42:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:41 smithi193 ceph-mon[21944]: pgmap v502: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 301 op/s 2024-08-20T06:42:41.661 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:42:42.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 -- 172.21.15.107:0/3958950183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10b050 msgr2=0x7f632c111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.192 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 --2- 172.21.15.107:0/3958950183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10b050 0x7f632c111850 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6324009ca0 tx=0x7f632402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 -- 172.21.15.107:0/3958950183 shutdown_connections 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 --2- 172.21.15.107:0/3958950183 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10b050 0x7f632c111850 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 --2- 172.21.15.107:0/3958950183 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c10a6d0 0x7f632c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 --2- 172.21.15.107:0/3958950183 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c104cc0 0x7f632c1050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 -- 172.21.15.107:0/3958950183 >> 172.21.15.107:0/3958950183 conn(0x7f632c1006f0 msgr2=0x7f632c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 -- 172.21.15.107:0/3958950183 shutdown_connections 2024-08-20T06:42:42.193 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.191+0000 7f63336ef640 1 -- 172.21.15.107:0/3958950183 wait complete. 2024-08-20T06:42:42.194 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 Processor -- start 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 -- start start 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10a6d0 0x7f632c1104a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c10b050 0x7f632c1ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f632c078310 con 0x7f632c10a6d0 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f632c078190 con 0x7f632c104cc0 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f63336ef640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f632c078490 con 0x7f632c10b050 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f633148b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f633148b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:57240/0 (socket says 172.21.15.107:57240) 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f633148b640 1 -- 172.21.15.107:0/2723314666 learned_addr learned my addr 172.21.15.107:0/2723314666 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10a6d0 0x7f632c1104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.192+0000 7f6331c8c640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c10b050 0x7f632c1ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 -- 172.21.15.107:0/2723314666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c10b050 msgr2=0x7f632c1ab110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c10b050 0x7f632c1ab110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 -- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10a6d0 msgr2=0x7f632c1104a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10a6d0 0x7f632c1104a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 -- 172.21.15.107:0/2723314666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c1ab650 con 0x7f632c104cc0 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f633148b640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9f0 tx=0x7f632800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f63227fc640 1 -- 172.21.15.107:0/2723314666 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632800cd80 con 0x7f632c104cc0 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f63227fc640 1 -- 172.21.15.107:0/2723314666 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63280108c0 con 0x7f632c104cc0 2024-08-20T06:42:42.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f63227fc640 1 -- 172.21.15.107:0/2723314666 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280187c0 con 0x7f632c104cc0 2024-08-20T06:42:42.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c1ab940 con 0x7f632c104cc0 2024-08-20T06:42:42.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.193+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c1abd70 con 0x7f632c104cc0 2024-08-20T06:42:42.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.196+0000 7f63227fc640 1 -- 172.21.15.107:0/2723314666 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f6328010430 con 0x7f632c104cc0 2024-08-20T06:42:42.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 Processor -- start 2024-08-20T06:42:42.198 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 -- start start 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d94fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d95bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d95e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d8c68e0 con 0x55ca4d950150 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d8b5de0 con 0x55ca4d94f830 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6335ae8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d8b9e20 con 0x55ca4d95c0e0 2024-08-20T06:42:42.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f633148b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d95bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f633148b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d95bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37116/0 (socket says 172.21.15.107:37116) 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f633148b640 1 -- 172.21.15.107:0/2597883064 learned_addr learned my addr 172.21.15.107:0/2597883064 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d95e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.197+0000 7f6331c8c640 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d94fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 -- 172.21.15.107:0/2597883064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 msgr2=0x55ca4d94fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d94fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 -- 172.21.15.107:0/2597883064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 msgr2=0x55ca4d95bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d95bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 -- 172.21.15.107:0/2597883064 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4d615650 con 0x55ca4d95c0e0 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f63227fc640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 0x7f63180b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d95e4d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f631c009c20 tx=0x7f631c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31a3f4df3fe9f93 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f6331c8c640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 0x7f63180b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.198+0000 7f63177fe640 1 -- 172.21.15.107:0/2597883064 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c004050 con 0x55ca4d95c0e0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f63177fe640 1 -- 172.21.15.107:0/2597883064 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f631c005020 con 0x55ca4d95c0e0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f63177fe640 1 -- 172.21.15.107:0/2597883064 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c004c30 con 0x55ca4d95c0e0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f63227fc640 1 -- 172.21.15.107:0/2723314666 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63280e3fb0 con 0x7f632c104cc0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f6335ae8740 1 -- 172.21.15.107:0/2597883064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 msgr2=0x55ca4d95e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f6331c8c640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 0x7f63180b6090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6324006010 tx=0x7f6324004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.199+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d95e4d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f631c009c20 tx=0x7f631c011810 comp rx=0 tx=0).stop 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 -- 172.21.15.107:0/2597883064 shutdown_connections 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d95e4d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d95bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2597883064 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d94fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 -- 172.21.15.107:0/2597883064 >> 172.21.15.107:0/2597883064 conn(0x55ca4d94d660 msgr2=0x55ca4d94e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 -- 172.21.15.107:0/2597883064 shutdown_connections 2024-08-20T06:42:42.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 -- 172.21.15.107:0/2597883064 wait complete. 2024-08-20T06:42:42.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 -- start start 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.200+0000 7f6335ae8740 1 Processor -- start 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d959ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d718700 con 0x55ca4d950150 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d83c7c0 con 0x55ca4d94f830 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ca4d95ed90 con 0x55ca4d95c0e0 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6335ae8740 10 client.0 _ll_register_callbacks cb 0x55ca4d8af6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6330c8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:57266/0 (socket says 172.21.15.107:57266) 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 -- 172.21.15.107:0/426280929 learned_addr learned my addr 172.21.15.107:0/426280929 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6330c8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9595a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37124/0 (socket says 172.21.15.107:37124) 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6331c8c640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d959ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 -- 172.21.15.107:0/426280929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 msgr2=0x55ca4d959ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d959ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 -- 172.21.15.107:0/426280929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 msgr2=0x55ca4d9595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 -- 172.21.15.107:0/426280929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4d968140 con 0x55ca4d94f830 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6331c8c640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d959ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f633148b640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63280e4250 tx=0x7f63280e1a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b613a6deb015216b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.201+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280aa6e0 con 0x55ca4d94f830 2024-08-20T06:42:42.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca4d865100 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63280e7b60 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6328010720 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6335ae8740 10 client.25721 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca4d96ab20 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ca4d865100 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.202+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63280e7020 con 0x55ca4d94f830 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 10 client.25721 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 10 client.25721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 20 client.25721 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d96e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 4 client.25721 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca4d8c8830 con 0x55ca4d96c280 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6330c8a640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d96e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6314ff9640 10 client.25721 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.203+0000 7f6330c8a640 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d96e670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=449932bdb940f03d server_cookie=2f09428155b6317b in_seq=0 out_seq=0 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6314ff9640 1 -- 172.21.15.107:0/426280929 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ca4d8c8830 con 0x55ca4d96c280 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6314ff9640 10 client.25721 handle_command_reply: tid=0 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 client.25721 shutdown 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 msgr2=0x55ca4d959060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63280e4250 tx=0x7f63280e1a30 comp rx=0 tx=0).stop 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 shutdown_connections 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d96e670 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d959ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6314ff9640 0 client.25721 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 --2- 172.21.15.107:0/426280929 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d94f830 0x55ca4d959060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.204+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 >> 172.21.15.107:0/426280929 conn(0x55ca4d94d660 msgr2=0x55ca4d953200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.205+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 shutdown_connections 2024-08-20T06:42:42.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.205+0000 7f6335ae8740 1 -- 172.21.15.107:0/426280929 wait complete. 2024-08-20T06:42:42.210 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.205+0000 7f6335ae8740 20 client.25721 trim_cache size 0 max 16384 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.232+0000 7f6335ae8740 1 Processor -- start 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.232+0000 7f6335ae8740 1 -- start start 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d94f830 0x55ca4d9538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d95c0e0 0x55ca4d973c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f632c1ae6e0 con 0x55ca4d950150 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f632c1abfd0 con 0x55ca4d95c0e0 2024-08-20T06:42:42.234 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6335ae8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f632c11a1a0 con 0x55ca4d94f830 2024-08-20T06:42:42.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6331c8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d95c0e0 0x55ca4d973c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37126/0 (socket says 172.21.15.107:37126) 2024-08-20T06:42:42.235 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 -- 172.21.15.107:0/266649989 learned_addr learned my addr 172.21.15.107:0/266649989 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f633148b640 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d94f830 0x55ca4d9538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 -- 172.21.15.107:0/266649989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d94f830 msgr2=0x55ca4d9538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d94f830 0x55ca4d9538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 -- 172.21.15.107:0/266649989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d95c0e0 msgr2=0x55ca4d973c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d95c0e0 0x55ca4d973c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.233+0000 7f6330c8a640 1 -- 172.21.15.107:0/266649989 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca4d94ff80 con 0x55ca4d950150 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.234+0000 7f6330c8a640 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f631c0099b0 tx=0x7f631c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba8430698f159a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.234+0000 7f6316ffd640 1 -- 172.21.15.107:0/266649989 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c046070 con 0x55ca4d950150 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.234+0000 7f6316ffd640 1 -- 172.21.15.107:0/266649989 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f631c005a70 con 0x55ca4d950150 2024-08-20T06:42:42.236 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.234+0000 7f6316ffd640 1 -- 172.21.15.107:0/266649989 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c003df0 con 0x55ca4d950150 2024-08-20T06:42:42.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 -- 172.21.15.107:0/266649989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 msgr2=0x55ca4d9596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f631c0099b0 tx=0x7f631c005f90 comp rx=0 tx=0).stop 2024-08-20T06:42:42.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 -- 172.21.15.107:0/266649989 shutdown_connections 2024-08-20T06:42:42.237 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d95c0e0 0x55ca4d973c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d950150 0x55ca4d9596e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 --2- 172.21.15.107:0/266649989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d94f830 0x55ca4d9538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 -- 172.21.15.107:0/266649989 >> 172.21.15.107:0/266649989 conn(0x55ca4d94d660 msgr2=0x55ca4d958fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 -- 172.21.15.107:0/266649989 shutdown_connections 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.235+0000 7f6335ae8740 1 -- 172.21.15.107:0/266649989 wait complete. 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.236+0000 7f6335ae8740 1 -- start start 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.236+0000 7f6335ae8740 1 Processor -- start 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.236+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.238 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d950150 0x55ca4d95fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d960120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f632c11a830 con 0x55ca4d94f830 2024-08-20T06:42:42.239 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f632c11bca0 con 0x55ca4d950150 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f632c11af00 con 0x55ca4d95c0e0 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6335ae8740 10 client.0 _ll_register_callbacks cb 0x7f632800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37138/0 (socket says 172.21.15.107:37138) 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 -- 172.21.15.107:0/2782266596 learned_addr learned my addr 172.21.15.107:0/2782266596 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d950150 0x55ca4d95fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f6331c8c640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d960120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 -- 172.21.15.107:0/2782266596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 msgr2=0x55ca4d960120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d960120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 -- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d950150 msgr2=0x55ca4d95fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d950150 0x55ca4d95fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.237+0000 7f633148b640 1 -- 172.21.15.107:0/2782266596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca4d960950 con 0x55ca4d94f830 2024-08-20T06:42:42.240 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f633148b640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f63280a7510 tx=0x7f63280aa480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f07b0eae7eb932 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6314ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63280aa6e0 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63280e7ae0 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631c005a70 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63280aad40 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6335ae8740 10 client.16212 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ca4d9d6cb0 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.238+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f631c005a70 con 0x55ca4d94f830 2024-08-20T06:42:42.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f63280b0650 con 0x55ca4d94f830 2024-08-20T06:42:42.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 10 client.16212 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:42.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:42:42.242 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 20 client.16212 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d9d9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 4 client.16212 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca4d8c42d0 con 0x55ca4d96c280 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d9d9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6330c8a640 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d9d9330 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29182eb140e2bf9d server_cookie=f65cff3a4d808541 in_seq=0 out_seq=0 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.239+0000 7f6314ff9640 10 client.16212 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6314ff9640 1 -- 172.21.15.107:0/2782266596 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ca4d8c42d0 con 0x55ca4d96c280 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6314ff9640 10 client.16212 handle_command_reply: tid=0 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 client.16212 shutdown 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 msgr2=0x55ca4d9646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f63280a7510 tx=0x7f63280aa480 comp rx=0 tx=0).stop 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 shutdown_connections 2024-08-20T06:42:42.243 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ca4d96c280 0x55ca4d9d9330 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ca4d95c0e0 0x55ca4d960120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ca4d950150 0x55ca4d95fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 --2- 172.21.15.107:0/2782266596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ca4d94f830 0x55ca4d9646c0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 >> 172.21.15.107:0/2782266596 conn(0x55ca4d94d660 msgr2=0x55ca4d959620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6314ff9640 0 client.16212 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 shutdown_connections 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 1 -- 172.21.15.107:0/2782266596 wait complete. 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.241+0000 7f6335ae8740 20 client.16212 trim_cache size 0 max 16384 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:42:42.244 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 msgr2=0x7f63180b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 0x7f63180b6090 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6324006010 tx=0x7f6324004a80 comp rx=0 tx=0).stop 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 msgr2=0x7f632c10ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:42.245 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f632800e9f0 tx=0x7f632800eeb0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 shutdown_connections 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63180b3bd0 0x7f63180b6090 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f632c10b050 0x7f632c1ab110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f632c10a6d0 0x7f632c1104a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 --2- 172.21.15.107:0/2723314666 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f632c104cc0 0x7f632c10ff60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.242+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 >> 172.21.15.107:0/2723314666 conn(0x7f632c1006f0 msgr2=0x7f632c102740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.243+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 shutdown_connections 2024-08-20T06:42:42.246 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:42.243+0000 7f63336ef640 1 -- 172.21.15.107:0/2723314666 wait complete. 2024-08-20T06:42:43.180 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:42:43.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:42:43.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:43 smithi107 ceph-mon[18495]: pgmap v503: 97 pgs: 97 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-08-20T06:42:43.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:43 smithi183 ceph-mon[22204]: pgmap v503: 97 pgs: 97 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-08-20T06:42:43.527 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:42:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:43 smithi193 ceph-mon[21944]: pgmap v503: 97 pgs: 97 active+clean; 390 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 308 op/s 2024-08-20T06:42:44.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.161+0000 7f84c62a7640 1 -- 172.21.15.107:0/2073969117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c01073a0 msgr2=0x7f84c01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.161+0000 7f84c62a7640 1 --2- 172.21.15.107:0/2073969117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c01073a0 0x7f84c01077e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f84b4009900 tx=0x7f84b402a410 comp rx=0 tx=0).stop 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 -- 172.21.15.107:0/2073969117 shutdown_connections 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 --2- 172.21.15.107:0/2073969117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 --2- 172.21.15.107:0/2073969117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c01073a0 0x7f84c01077e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 --2- 172.21.15.107:0/2073969117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c0101990 0x7f84c0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 -- 172.21.15.107:0/2073969117 >> 172.21.15.107:0/2073969117 conn(0x7f84c00fd400 msgr2=0x7f84c00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 -- 172.21.15.107:0/2073969117 shutdown_connections 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 -- 172.21.15.107:0/2073969117 wait complete. 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.162+0000 7f84c62a7640 1 Processor -- start 2024-08-20T06:42:44.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.163+0000 7f84c62a7640 1 -- start start 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.163+0000 7f84c62a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 0x7f84c0198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.163+0000 7f84c62a7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c01073a0 0x7f84c01994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.163+0000 7f84c62a7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c019e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.163+0000 7f84c62a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01118a0 con 0x7f84c0101990 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 0x7f84c0198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84bf7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c01073a0 0x7f84c01994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:44.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84bffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 0x7f84c0198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37150/0 (socket says 172.21.15.107:37150) 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84bffff640 1 -- 172.21.15.107:0/738614661 learned_addr learned my addr 172.21.15.107:0/738614661 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c62a7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0111720 con 0x7f84c01073a0 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f84c0111a20 con 0x7f84c0107d20 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c019e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 -- 172.21.15.107:0/738614661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c01073a0 msgr2=0x7f84c01994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c01073a0 0x7f84c01994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 -- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 msgr2=0x7f84c0198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 0x7f84c0198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c019f010 con 0x7f84c0107d20 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.164+0000 7f84c4844640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c019e840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009fd0 tx=0x7f84b800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.165+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b800ce20 con 0x7f84c0107d20 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.165+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f84b8005020 con 0x7f84c0107d20 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.165+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c019f300 con 0x7f84c0107d20 2024-08-20T06:42:44.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.165+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a5b90 con 0x7f84c0107d20 2024-08-20T06:42:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.166+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8010690 con 0x7f84c0107d20 2024-08-20T06:42:44.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.166+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84c0101d70 con 0x7f84c0107d20 2024-08-20T06:42:44.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.168+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f84b8004b20 con 0x7f84c0107d20 2024-08-20T06:42:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.169+0000 7f84bd7fa640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 0x7f84a80b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.169+0000 7f84bffff640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 0x7f84a80b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.169+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f84b80e4130 con 0x7f84c0107d20 2024-08-20T06:42:44.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.169+0000 7f84bffff640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 0x7f84a80b6110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009c90 tx=0x7f84b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:44.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.173+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f84b80ab780 con 0x7f84c0107d20 2024-08-20T06:42:44.507 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.507+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84c01a5d60 con 0x7f84c0107d20 2024-08-20T06:42:44.508 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.508+0000 7f84bd7fa640 1 -- 172.21.15.107:0/738614661 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7f84b80ab080 con 0x7f84c0107d20 2024-08-20T06:42:44.509 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:42:44.509 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":29,"btime":"2024-08-20T06:41:46:295359+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:41:45.321550+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:42:44.509 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 29 2024-08-20T06:42:44.511 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.510+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 msgr2=0x7f84a80b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.510+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 0x7f84a80b6110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009c90 tx=0x7f84b0005f90 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.510+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 msgr2=0x7f84c019e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c019e840 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009fd0 tx=0x7f84b800ef10 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 shutdown_connections 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f84a80b3c50 0x7f84a80b6110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84c0107d20 0x7f84c019e840 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f84c01073a0 0x7f84c01994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 --2- 172.21.15.107:0/738614661 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f84c0101990 0x7f84c0198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 >> 172.21.15.107:0/738614661 conn(0x7f84c00fd400 msgr2=0x7f84c01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 shutdown_connections 2024-08-20T06:42:44.512 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:44.511+0000 7f84c62a7640 1 -- 172.21.15.107:0/738614661 wait complete. 2024-08-20T06:42:45.133 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:45 smithi107 ceph-mon[18495]: pgmap v504: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-08-20T06:42:45.133 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:45 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/738614661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:42:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:45 smithi183 ceph-mon[22204]: pgmap v504: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-08-20T06:42:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:45 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/738614661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:42:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:45 smithi193 ceph-mon[21944]: pgmap v504: 97 pgs: 97 active+clean; 392 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-08-20T06:42:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:45 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/738614661' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:42:45.887 INFO:tasks.workunit.client.0.smithi107.stdout: 86 12257 116 8045 143 5608 281 2024-08-20T06:42:47.356 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:47 smithi193 ceph-mon[21944]: pgmap v505: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-08-20T06:42:47.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:47 smithi107 ceph-mon[18495]: pgmap v505: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-08-20T06:42:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:47 smithi183 ceph-mon[22204]: pgmap v505: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 270 op/s 2024-08-20T06:42:49.273 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:42:49.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:49 smithi107 ceph-mon[18495]: pgmap v506: 97 pgs: 97 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-08-20T06:42:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:49 smithi183 ceph-mon[22204]: pgmap v506: 97 pgs: 97 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-08-20T06:42:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:49 smithi193 ceph-mon[21944]: pgmap v506: 97 pgs: 97 active+clean; 394 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 286 op/s 2024-08-20T06:42:49.662 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:42:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:51 smithi107 ceph-mon[18495]: pgmap v507: 97 pgs: 97 active+clean; 391 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-08-20T06:42:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:51 smithi183 ceph-mon[22204]: pgmap v507: 97 pgs: 97 active+clean; 391 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-08-20T06:42:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:51 smithi193 ceph-mon[21944]: pgmap v507: 97 pgs: 97 active+clean; 391 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.6 MiB/s wr, 264 op/s 2024-08-20T06:42:53.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:53 smithi107 ceph-mon[18495]: pgmap v508: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2024-08-20T06:42:53.485 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:53 smithi183 ceph-mon[22204]: pgmap v508: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2024-08-20T06:42:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:53 smithi193 ceph-mon[21944]: pgmap v508: 97 pgs: 97 active+clean; 388 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.3 MiB/s wr, 275 op/s 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.220+0000 7f2b206b8640 1 -- 172.21.15.107:0/3731847509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 msgr2=0x7f2b18101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.220+0000 7f2b206b8640 1 --2- 172.21.15.107:0/3731847509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b18101d70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009900 tx=0x7f2b1402a410 comp rx=0 tx=0).stop 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 -- 172.21.15.107:0/3731847509 shutdown_connections 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 --2- 172.21.15.107:0/3731847509 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b18107d20 0x7f2b1810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 --2- 172.21.15.107:0/3731847509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b181073a0 0x7f2b181077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 --2- 172.21.15.107:0/3731847509 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b18101d70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.113 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 -- 172.21.15.107:0/3731847509 >> 172.21.15.107:0/3731847509 conn(0x7f2b180fd400 msgr2=0x7f2b180ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 -- 172.21.15.107:0/3731847509 shutdown_connections 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 -- 172.21.15.107:0/3731847509 wait complete. 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.221+0000 7f2b206b8640 1 Processor -- start 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 -- start start 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b1810d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b18107d20 0x7f2b181a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181129b0 con 0x7f2b18101990 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18112830 con 0x7f2b181073a0 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b206b8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18112b30 con 0x7f2b18107d20 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.114 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51408/0 (socket says 172.21.15.107:51408) 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 -- 172.21.15.107:0/185966347 learned_addr learned my addr 172.21.15.107:0/185966347 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b18107d20 0x7f2b181a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1e454640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b1810d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 -- 172.21.15.107:0/185966347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b18107d20 msgr2=0x7f2b181a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b18107d20 0x7f2b181a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 -- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 msgr2=0x7f2b1810d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b1810d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1dc53640 1 -- 172.21.15.107:0/185966347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b181a55a0 con 0x7f2b181073a0 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.222+0000 7f2b1e454640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b1810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9f0 tx=0x7f2b0800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.115 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b0f7fe640 1 -- 172.21.15.107:0/185966347 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800cd80 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b181a5890 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b0f7fe640 1 -- 172.21.15.107:0/185966347 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b080108c0 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b0f7fe640 1 -- 172.21.15.107:0/185966347 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b080187c0 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.223+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181abfa0 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.226+0000 7f2b0f7fe640 1 -- 172.21.15.107:0/185966347 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f2b08010430 con 0x7f2b181073a0 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.226+0000 7f2b22ab1740 1 Processor -- start 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 -- start start 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.116 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 0x55f487e4fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487e52520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f487dba930 con 0x55f487e43880 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f487da9e30 con 0x55f487e441a0 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f487dade70 con 0x55f487e50130 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b0f7fe640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 0x7f2b040b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1ec55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1ec55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:38398/0 (socket says 172.21.15.107:38398) 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1ec55640 1 -- 172.21.15.107:0/4129328906 learned_addr learned my addr 172.21.15.107:0/4129328906 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487e52520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.117 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1e454640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 0x55f487e4fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.227+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 0x7f2b040b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b0f7fe640 1 -- 172.21.15.107:0/185966347 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b080e3ff0 con 0x7f2b181073a0 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 -- 172.21.15.107:0/4129328906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 msgr2=0x55f487e52520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487e52520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 -- 172.21.15.107:0/4129328906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 msgr2=0x55f487e4fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 0x55f487e4fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 -- 172.21.15.107:0/4129328906 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f487b09650 con 0x55f487e43880 2024-08-20T06:42:54.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1e454640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 0x55f487e4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 0x7f2b040b6110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b100099e0 tx=0x7f2b10009c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b100091e0 tx=0x7f2b1001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64e2e84946afe2c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b0cff9640 1 -- 172.21.15.107:0/4129328906 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10022070 con 0x55f487e43880 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b0cff9640 1 -- 172.21.15.107:0/4129328906 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b1000ee50 con 0x55f487e43880 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.228+0000 7f2b0cff9640 1 -- 172.21.15.107:0/4129328906 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004290 con 0x55f487e43880 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 -- 172.21.15.107:0/4129328906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 msgr2=0x55f487e43c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2b100091e0 tx=0x7f2b1001d000 comp rx=0 tx=0).stop 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 -- 172.21.15.107:0/4129328906 shutdown_connections 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487e52520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e441a0 0x55f487e4fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/4129328906 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e43880 0x55f487e43c60 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.229+0000 7f2b22ab1740 1 -- 172.21.15.107:0/4129328906 >> 172.21.15.107:0/4129328906 conn(0x55f487e416b0 msgr2=0x55f487e42850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- 172.21.15.107:0/4129328906 shutdown_connections 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- 172.21.15.107:0/4129328906 wait complete. 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- start start 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 Processor -- start 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e50130 0x55f487e4dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55f487c0c700 con 0x55f487e441a0 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55f487d30810 con 0x55f487e50130 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55f487e52d80 con 0x55f487e43880 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.230+0000 7f2b22ab1740 10 client.0 _ll_register_callbacks cb 0x55f487da3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:54674/0 (socket says 172.21.15.107:54674) 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 -- 172.21.15.107:0/3488214469 learned_addr learned my addr 172.21.15.107:0/3488214469 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e50130 0x55f487e4dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 -- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e50130 msgr2=0x55f487e4dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e50130 0x55f487e4dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 -- 172.21.15.107:0/3488214469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 msgr2=0x55f487e4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 -- 172.21.15.107:0/3488214469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f487e5c1a0 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b1e454640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b1402aa00 tx=0x7f2b14005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1ff95a51d77640e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14035040 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f487d59150 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.231+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b140046c0 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b14035040 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 10 client.25633 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f487e5eba0 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55f487d59150 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b14003ab0 con 0x55f487e43880 2024-08-20T06:42:54.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 10 client.25633 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 10 client.25633 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 20 client.25633 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487e623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 4 client.25633 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.232+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f487dbc880 con 0x55f487e5ffb0 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.233+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487e623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.233+0000 7f2b01ffb640 10 client.25633 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.233+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487e623e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d36eed82619dd897 server_cookie=8ad2e0e68a0d5012 in_seq=0 out_seq=0 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b01ffb640 1 -- 172.21.15.107:0/3488214469 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f487dbc880 con 0x55f487e5ffb0 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b01ffb640 10 client.25633 handle_command_reply: tid=0 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 client.25633 shutdown 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 msgr2=0x55f487e4d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2b1402aa00 tx=0x7f2b14005270 comp rx=0 tx=0).stop 2024-08-20T06:42:54.124 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 shutdown_connections 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487e623e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b01ffb640 0 client.25633 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e50130 0x55f487e4dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/3488214469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e43880 0x55f487e4d180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 >> 172.21.15.107:0/3488214469 conn(0x55f487e416b0 msgr2=0x55f487e47250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 shutdown_connections 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.234+0000 7f2b22ab1740 1 -- 172.21.15.107:0/3488214469 wait complete. 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.235+0000 7f2b22ab1740 20 client.25633 trim_cache size 0 max 16384 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 Processor -- start 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 -- start start 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.125 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e441a0 0x55f487e681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e50130 0x55f487e57fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b6f20 con 0x55f487e50130 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b47f0 con 0x55f487e43880 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b2620 con 0x55f487e441a0 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b1ec55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e50130 0x55f487e57fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51428/0 (socket says 172.21.15.107:51428) 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b1e454640 1 -- 172.21.15.107:0/955382173 learned_addr learned my addr 172.21.15.107:0/955382173 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.254+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e441a0 0x55f487e681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 -- 172.21.15.107:0/955382173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e441a0 msgr2=0x55f487e681f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e441a0 0x55f487e681f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 -- 172.21.15.107:0/955382173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e50130 msgr2=0x55f487e57fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.126 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e50130 0x55f487e57fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 -- 172.21.15.107:0/955382173 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f487e43fd0 con 0x55f487e43880 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b1e454640 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b1402a450 tx=0x7f2b14045470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=568d6ba1e2f66235 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b03fff640 1 -- 172.21.15.107:0/955382173 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b140458e0 con 0x55f487e43880 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b03fff640 1 -- 172.21.15.107:0/955382173 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b14040070 con 0x55f487e43880 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.255+0000 7f2b03fff640 1 -- 172.21.15.107:0/955382173 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14003a50 con 0x55f487e43880 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- 172.21.15.107:0/955382173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 msgr2=0x55f487e67cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2b1402a450 tx=0x7f2b14045470 comp rx=0 tx=0).stop 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- 172.21.15.107:0/955382173 shutdown_connections 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e50130 0x55f487e57fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e441a0 0x55f487e681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/955382173 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e67cb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- 172.21.15.107:0/955382173 >> 172.21.15.107:0/955382173 conn(0x55f487e416b0 msgr2=0x55f487e4efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- 172.21.15.107:0/955382173 shutdown_connections 2024-08-20T06:42:54.127 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- 172.21.15.107:0/955382173 wait complete. 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 -- start start 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.256+0000 7f2b22ab1740 1 Processor -- start 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e55aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487ec8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b2cb0 con 0x55f487e441a0 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b4120 con 0x55f487e43880 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181b3380 con 0x55f487e50130 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b22ab1740 10 client.0 _ll_register_callbacks cb 0x7f2b08005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1ec55640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487ec8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1e454640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51434/0 (socket says 172.21.15.107:51434) 2024-08-20T06:42:54.128 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1e454640 1 -- 172.21.15.107:0/724555466 learned_addr learned my addr 172.21.15.107:0/724555466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e55aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1ec55640 1 -- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 msgr2=0x55f487e55560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1ec55640 1 -- 172.21.15.107:0/724555466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 msgr2=0x55f487e55aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.257+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e55aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b1ec55640 1 -- 172.21.15.107:0/724555466 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f487ec9210 con 0x55f487e50130 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b1e454640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b1dc53640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e55aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b1ec55640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487ec8c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003e80 tx=0x7f2b10011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e155a415f5bc27fc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10036060 con 0x55f487e50130 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f2b100050b0 con 0x55f487e50130 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b14040070 con 0x55f487e50130 2024-08-20T06:42:54.129 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2b10022040 con 0x55f487e50130 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b22ab1740 10 client.25639 fetch_fsmap learned FSMap version 29 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.258+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f487eca510 con 0x55f487e50130 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.259+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2b14040070 con 0x55f487e50130 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.259+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2b10031480 con 0x55f487e50130 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.259+0000 7f2b22ab1740 10 client.25639 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b22ab1740 10 client.25639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:42:54.130 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b22ab1740 20 client.25639 populate_metadata read hostname 'smithi107' 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487ecca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b22ab1740 4 client.25639 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f487e47ea0 con 0x55f487e5ffb0 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b1e454640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487ecca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b1e454640 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487ecca90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94f152388ad71227 server_cookie=cc093bf8345e9f77 in_seq=0 out_seq=0 2024-08-20T06:42:54.131 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.260+0000 7f2b01ffb640 10 client.25639 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b01ffb640 1 -- 172.21.15.107:0/724555466 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f487e47ea0 con 0x55f487e5ffb0 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b01ffb640 10 client.25639 handle_command_reply: tid=0 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 client.25639 shutdown 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 msgr2=0x55f487ec8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487ec8c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2b10003e80 tx=0x7f2b10011440 comp rx=0 tx=0).stop 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 shutdown_connections 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f487e50130 0x55f487ec8c80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55f487e441a0 0x55f487e55aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55f487e5ffb0 0x55f487ecca90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b01ffb640 0 client.25639 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 --2- 172.21.15.107:0/724555466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55f487e43880 0x55f487e55560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.132 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 >> 172.21.15.107:0/724555466 conn(0x55f487e416b0 msgr2=0x55f487e4e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 shutdown_connections 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.261+0000 7f2b22ab1740 1 -- 172.21.15.107:0/724555466 wait complete. 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b22ab1740 20 client.25639 trim_cache size 0 max 16384 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 msgr2=0x7f2b040b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 0x7f2b040b6110 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2b100099e0 tx=0x7f2b10009c80 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 msgr2=0x7f2b1810de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9f0 tx=0x7f2b0800eeb0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 shutdown_connections 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2b040b3c50 0x7f2b040b6110 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b18107d20 0x7f2b181a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2b181073a0 0x7f2b1810de60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 --2- 172.21.15.107:0/185966347 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b18101990 0x7f2b1810d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:42:54.133 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.262+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 >> 172.21.15.107:0/185966347 conn(0x7f2b180fd400 msgr2=0x7f2b18103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:42:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.263+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 shutdown_connections 2024-08-20T06:42:54.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:42:53.263+0000 7f2b206b8640 1 -- 172.21.15.107:0/185966347 wait complete. 2024-08-20T06:42:55.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:55 smithi107 ceph-mon[18495]: pgmap v509: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-20T06:42:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:55 smithi183 ceph-mon[22204]: pgmap v509: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-20T06:42:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:55 smithi193 ceph-mon[21944]: pgmap v509: 97 pgs: 97 active+clean; 389 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-08-20T06:42:55.887 INFO:tasks.workunit.client.0.smithi107.stdout: 87 10797 111 6968 93 5841 225 2024-08-20T06:42:57.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:57 smithi107 ceph-mon[18495]: pgmap v510: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-08-20T06:42:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:57 smithi183 ceph-mon[22204]: pgmap v510: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-08-20T06:42:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:57 smithi193 ceph-mon[21944]: pgmap v510: 97 pgs: 97 active+clean; 397 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 235 op/s 2024-08-20T06:42:59.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:42:59 smithi107 ceph-mon[18495]: pgmap v511: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2024-08-20T06:42:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:42:59 smithi183 ceph-mon[22204]: pgmap v511: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2024-08-20T06:42:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:42:59 smithi193 ceph-mon[21944]: pgmap v511: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 239 op/s 2024-08-20T06:43:01.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:01 smithi107 ceph-mon[18495]: pgmap v512: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-08-20T06:43:01.362 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:43:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:01 smithi183 ceph-mon[22204]: pgmap v512: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-08-20T06:43:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:01 smithi193 ceph-mon[21944]: pgmap v512: 97 pgs: 97 active+clean; 398 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 230 op/s 2024-08-20T06:43:01.735 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:02.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 -- 172.21.15.107:0/761269146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741039a0 msgr2=0x7f8a74103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/761269146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741039a0 0x7f8a74103d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8a64009900 tx=0x7f8a6402a410 comp rx=0 tx=0).stop 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 -- 172.21.15.107:0/761269146 shutdown_connections 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/761269146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 0x7f8a7410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/761269146 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741042c0 0x7f8a7410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/761269146 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741039a0 0x7f8a74103d80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 -- 172.21.15.107:0/761269146 >> 172.21.15.107:0/761269146 conn(0x7f8a740fd400 msgr2=0x7f8a740ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 -- 172.21.15.107:0/761269146 shutdown_connections 2024-08-20T06:43:02.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.253+0000 7f8a78a8c640 1 -- 172.21.15.107:0/761269146 wait complete. 2024-08-20T06:43:02.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.254+0000 7f8a78a8c640 1 Processor -- start 2024-08-20T06:43:02.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.254+0000 7f8a78a8c640 1 -- start start 2024-08-20T06:43:02.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741042c0 0x7f8a74199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 0x7f8a7419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74112970 con 0x7f8a741042c0 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741127f0 con 0x7f8a741039a0 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a78a8c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a74112af0 con 0x7f8a7410ce00 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51872/0 (socket says 172.21.15.107:51872) 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 -- 172.21.15.107:0/3307167547 learned_addr learned my addr 172.21.15.107:0/3307167547 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741042c0 0x7f8a74199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 0x7f8a7419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 -- 172.21.15.107:0/3307167547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 msgr2=0x7f8a7419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 0x7f8a7419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 -- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741042c0 msgr2=0x7f8a74199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741042c0 0x7f8a74199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.255+0000 7f8a7259c640 1 -- 172.21.15.107:0/3307167547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7419edf0 con 0x7f8a741039a0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402a930 tx=0x7f8a64030650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a637fe640 1 -- 172.21.15.107:0/3307167547 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6403f070 con 0x7f8a741039a0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7419f080 con 0x7f8a741039a0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a637fe640 1 -- 172.21.15.107:0/3307167547 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a64004460 con 0x7f8a741039a0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a637fe640 1 -- 172.21.15.107:0/3307167547 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64035030 con 0x7f8a741039a0 2024-08-20T06:43:02.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.256+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a741a57a0 con 0x7f8a741039a0 2024-08-20T06:43:02.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.259+0000 7f8a637fe640 1 -- 172.21.15.107:0/3307167547 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f8a64042070 con 0x7f8a741039a0 2024-08-20T06:43:02.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.259+0000 7f8a7ae85740 1 Processor -- start 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 -- start start 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5646bc23b930 con 0x5646bc2c8c90 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646bc22ae30 con 0x5646bc2c95b0 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5646bc22ee70 con 0x5646bc2d5540 2024-08-20T06:43:02.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a72d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51878/0 (socket says 172.21.15.107:51878) 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 -- 172.21.15.107:0/2579170265 learned_addr learned my addr 172.21.15.107:0/2579170265 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a72d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55736/0 (socket says 172.21.15.107:55736) 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a637fe640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 0x7f8a5c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 -- 172.21.15.107:0/2579170265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 msgr2=0x5646bc2d7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.260+0000 7f8a7259c640 1 -- 172.21.15.107:0/2579170265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 msgr2=0x5646bc2c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a7259c640 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a7259c640 1 -- 172.21.15.107:0/2579170265 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646bbf8a650 con 0x5646bc2c95b0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 0x7f8a5c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a7259c640 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a640e4e20 tx=0x7f8a640c7990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4c668cfa9003748 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a637fe640 1 -- 172.21.15.107:0/3307167547 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8a641042c0 con 0x7f8a741039a0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a60ff9640 1 -- 172.21.15.107:0/2579170265 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64049030 con 0x5646bc2c95b0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a60ff9640 1 -- 172.21.15.107:0/2579170265 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a64030df0 con 0x5646bc2c95b0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.261+0000 7f8a60ff9640 1 -- 172.21.15.107:0/2579170265 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640cafe0 con 0x5646bc2c95b0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 0x7f8a5c0b6090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c005fd0 tx=0x7f8a6c004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 -- 172.21.15.107:0/2579170265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 msgr2=0x5646bc2d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8a640e4e20 tx=0x7f8a640c7990 comp rx=0 tx=0).stop 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 -- 172.21.15.107:0/2579170265 shutdown_connections 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2d7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c95b0 0x5646bc2d5000 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/2579170265 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 -- 172.21.15.107:0/2579170265 >> 172.21.15.107:0/2579170265 conn(0x5646bc2c6ac0 msgr2=0x5646bc2c7c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 -- 172.21.15.107:0/2579170265 shutdown_connections 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.262+0000 7f8a7ae85740 1 -- 172.21.15.107:0/2579170265 wait complete. 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.263+0000 7f8a7ae85740 1 -- start start 2024-08-20T06:43:02.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.263+0000 7f8a7ae85740 1 Processor -- start 2024-08-20T06:43:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.263+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 0x5646bc2d2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5646bc08d700 con 0x5646bc2c8c90 2024-08-20T06:43:02.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5646bc1b1810 con 0x5646bc2d5540 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5646bc2d7f80 con 0x5646bc2c95b0 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55744/0 (socket says 172.21.15.107:55744) 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7259c640 1 -- 172.21.15.107:0/3323321784 learned_addr learned my addr 172.21.15.107:0/3323321784 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7ae85740 10 client.0 _ll_register_callbacks cb 0x5646bc224700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 0x5646bc2d2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 -- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 msgr2=0x5646bc2d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 -- 172.21.15.107:0/3323321784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 msgr2=0x5646bc2d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 -- 172.21.15.107:0/3323321784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646bc2e1270 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 0x5646bc2d2f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009fd0 tx=0x7f8a6c0096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5ef5c8a93b561f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.264+0000 7f8a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6c012850 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8a6c0129f0 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646bc1da150 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a6c012bf0 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 10 client.25763 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646bc2e3c60 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x5646bc1da150 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8a6c01b070 con 0x5646bc2d5540 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 10 client.25763 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:02.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 10 client.25763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 20 client.25763 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2e5170 0x5646bc2e7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 4 client.25763 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.265+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646bc23d880 con 0x5646bc2e5170 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.266+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2e5170 0x5646bc2e7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.266+0000 7f8a7259c640 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2e5170 0x5646bc2e7590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a35cefa64781ecbe server_cookie=f068496adbba811 in_seq=0 out_seq=0 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.266+0000 7f8a59ffb640 10 client.25763 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a59ffb640 1 -- 172.21.15.107:0/3323321784 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5646bc23d880 con 0x5646bc2e5170 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a59ffb640 10 client.25763 handle_command_reply: tid=0 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 client.25763 shutdown 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 msgr2=0x5646bc2d2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 0x5646bc2d2f40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009fd0 tx=0x7f8a6c0096e0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 shutdown_connections 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2e5170 0x5646bc2e7590 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2d5540 0x5646bc2d2f40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/3323321784 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c8c90 0x5646bc2d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 >> 172.21.15.107:0/3323321784 conn(0x5646bc2c6ac0 msgr2=0x5646bc2c1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a59ffb640 0 client.25763 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 shutdown_connections 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 1 -- 172.21.15.107:0/3323321784 wait complete. 2024-08-20T06:43:02.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.267+0000 7f8a7ae85740 20 client.25763 trim_cache size 0 max 16384 2024-08-20T06:43:02.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 Processor -- start 2024-08-20T06:43:02.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 -- start start 2024-08-20T06:43:02.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741ae7e0 con 0x5646bc2c95b0 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741ac0b0 con 0x5646bc2c8c90 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741a9ee0 con 0x5646bc2d5540 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a71d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55760/0 (socket says 172.21.15.107:55760) 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a71d9b640 1 -- 172.21.15.107:0/1355542914 learned_addr learned my addr 172.21.15.107:0/1355542914 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.288+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 -- 172.21.15.107:0/1355542914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 msgr2=0x5646bc2ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 -- 172.21.15.107:0/1355542914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 msgr2=0x5646bc2ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 -- 172.21.15.107:0/1355542914 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646bc2d09e0 con 0x5646bc2c8c90 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a72d9d640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2c7eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402a9d0 tx=0x7f8a6400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e7314d130eb99f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a5bfff640 1 -- 172.21.15.107:0/1355542914 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64049040 con 0x5646bc2c8c90 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a5bfff640 1 -- 172.21.15.107:0/1355542914 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a6400b0b0 con 0x5646bc2c8c90 2024-08-20T06:43:02.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.289+0000 7f8a5bfff640 1 -- 172.21.15.107:0/1355542914 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640ca470 con 0x5646bc2c8c90 2024-08-20T06:43:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1355542914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 msgr2=0x5646bc2c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2c7eb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402a9d0 tx=0x7f8a6400b040 comp rx=0 tx=0).stop 2024-08-20T06:43:02.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1355542914 shutdown_connections 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2d5540 0x5646bc2ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2c95b0 0x5646bc2ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1355542914 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2c7eb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1355542914 >> 172.21.15.107:0/1355542914 conn(0x5646bc2c6ac0 msgr2=0x5646bc2c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1355542914 shutdown_connections 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1355542914 wait complete. 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.290+0000 7f8a7ae85740 1 -- start start 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 Processor -- start 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2d5540 0x5646bc2cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741aa570 con 0x5646bc2d5540 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741ab9e0 con 0x5646bc2c8c90 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8a741aac40 con 0x5646bc2c95b0 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7ae85740 10 client.0 _ll_register_callbacks cb 0x7f8a6c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a72d9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2d5540 0x5646bc2cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51908/0 (socket says 172.21.15.107:51908) 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 -- 172.21.15.107:0/1130776622 learned_addr learned my addr 172.21.15.107:0/1130776622 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 -- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 msgr2=0x5646bc2dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 -- 172.21.15.107:0/1130776622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2d5540 msgr2=0x5646bc2cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:02.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2d5540 0x5646bc2cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.291+0000 7f8a7259c640 1 -- 172.21.15.107:0/1130776622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646bc2cf380 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a7259c640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a640c8480 tx=0x7f8a64005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4539e60c2375f113 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a59ffb640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a640ea060 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8a640cb470 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a6400b0b0 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a64049040 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a7ae85740 10 client.25769 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646bc2c59e0 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.292+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8a640cb6f0 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8a6404c060 con 0x5646bc2c8c90 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:02.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 10 client.25769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 20 client.25769 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2c5f10 0x5646bc356630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 4 client.25769 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8a6400b0b0 con 0x5646bc2c5f10 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.293+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2c5f10 0x5646bc356630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a59ffb640 10 client.25769 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a71d9b640 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2c5f10 0x5646bc356630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aee541175e9697e2 server_cookie=d0e74b053e42e6dd in_seq=0 out_seq=0 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a59ffb640 1 -- 172.21.15.107:0/1130776622 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8a6400b0b0 con 0x5646bc2c5f10 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a59ffb640 10 client.25769 handle_command_reply: tid=0 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 client.25769 shutdown 2024-08-20T06:43:02.300 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 msgr2=0x5646bc2dd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8a640c8480 tx=0x7f8a64005fd0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 shutdown_connections 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x5646bc2c5f10 0x5646bc356630 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x5646bc2d5540 0x5646bc2cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5646bc2c95b0 0x5646bc2dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a59ffb640 0 client.25769 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 --2- 172.21.15.107:0/1130776622 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x5646bc2c8c90 0x5646bc2dd3b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.294+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 >> 172.21.15.107:0/1130776622 conn(0x5646bc2c6ac0 msgr2=0x5646bc2d4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 shutdown_connections 2024-08-20T06:43:02.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a7ae85740 1 -- 172.21.15.107:0/1130776622 wait complete. 2024-08-20T06:43:02.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a7ae85740 20 client.25769 trim_cache size 0 max 16384 2024-08-20T06:43:02.302 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 msgr2=0x7f8a5c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 0x7f8a5c0b6090 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c005fd0 tx=0x7f8a6c004ab0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 msgr2=0x7f8a74198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a6402a930 tx=0x7f8a64030650 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 shutdown_connections 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8a5c0b3bd0 0x7f8a5c0b6090 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a7410ce00 0x7f8a7419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a741042c0 0x7f8a74199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 --2- 172.21.15.107:0/3307167547 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8a741039a0 0x7f8a74198d40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:02.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.295+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 >> 172.21.15.107:0/3307167547 conn(0x7f8a740fd400 msgr2=0x7f8a74104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:02.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.296+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 shutdown_connections 2024-08-20T06:43:02.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:02.296+0000 7f8a78a8c640 1 -- 172.21.15.107:0/3307167547 wait complete. 2024-08-20T06:43:03.204 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:03 smithi107 ceph-mon[18495]: pgmap v513: 97 pgs: 97 active+clean; 397 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-08-20T06:43:03.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:43:03.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:03 smithi183 ceph-mon[22204]: pgmap v513: 97 pgs: 97 active+clean; 397 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-08-20T06:43:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:03 smithi193 ceph-mon[21944]: pgmap v513: 97 pgs: 97 active+clean; 397 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 240 op/s 2024-08-20T06:43:03.614 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 -- 172.21.15.107:0/2308261954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654109110 msgr2=0x7f36541094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.161 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 --2- 172.21.15.107:0/2308261954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654109110 0x7f36541094f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3644009900 tx=0x7f364402a410 comp rx=0 tx=0).stop 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 -- 172.21.15.107:0/2308261954 shutdown_connections 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 --2- 172.21.15.107:0/2308261954 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410f450 0x7f3654115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 --2- 172.21.15.107:0/2308261954 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f365410ead0 0x7f365410ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 --2- 172.21.15.107:0/2308261954 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3654109110 0x7f36541094f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 -- 172.21.15.107:0/2308261954 >> 172.21.15.107:0/2308261954 conn(0x7f3654078760 msgr2=0x7f365407ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 -- 172.21.15.107:0/2308261954 shutdown_connections 2024-08-20T06:43:04.162 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.160+0000 7f36595fa640 1 -- 172.21.15.107:0/2308261954 wait complete. 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.161+0000 7f36595fa640 1 Processor -- start 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.161+0000 7f36595fa640 1 -- start start 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.161+0000 7f36595fa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 0x7f365419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.161+0000 7f36595fa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36595fa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365410f450 0x7f36541a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36595fa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f3654118f60 con 0x7f365410f450 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36595fa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f3654118de0 con 0x7f365410ead0 2024-08-20T06:43:04.163 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51930/0 (socket says 172.21.15.107:51930) 2024-08-20T06:43:04.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36595fa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36541190e0 con 0x7f3654109110 2024-08-20T06:43:04.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 -- 172.21.15.107:0/3426859962 learned_addr learned my addr 172.21.15.107:0/3426859962 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:04.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36537fe640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365410f450 0x7f36541a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f3652ffd640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 0x7f365419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 -- 172.21.15.107:0/3426859962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 msgr2=0x7f365419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 0x7f365419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 -- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365410f450 msgr2=0x7f36541a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365410f450 0x7f36541a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f36527fc640 1 -- 172.21.15.107:0/3426859962 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36541a5660 con 0x7f365410ead0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.162+0000 7f3652ffd640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 0x7f365419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f36527fc640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f364800d950 tx=0x7f364800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f363bfff640 1 -- 172.21.15.107:0/3426859962 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648017070 con 0x7f365410ead0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f363bfff640 1 -- 172.21.15.107:0/3426859962 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3648009c30 con 0x7f365410ead0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36541a5950 con 0x7f365410ead0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36541ac010 con 0x7f365410ead0 2024-08-20T06:43:04.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.163+0000 7f363bfff640 1 -- 172.21.15.107:0/3426859962 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364800f620 con 0x7f365410ead0 2024-08-20T06:43:04.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.167+0000 7f363bfff640 1 -- 172.21.15.107:0/3426859962 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f3648010460 con 0x7f365410ead0 2024-08-20T06:43:04.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.167+0000 7f365b9f3740 1 Processor -- start 2024-08-20T06:43:04.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.167+0000 7f365b9f3740 1 -- start start 2024-08-20T06:43:04.168 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a86c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f363bfff640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 0x7f363c0b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a92bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a86c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36537fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a86c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55798/0 (socket says 172.21.15.107:55798) 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a95520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36537fe640 1 -- 172.21.15.107:0/3423911708 learned_addr learned my addr 172.21.15.107:0/3423911708 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f3652ffd640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 0x7f363c0b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a92bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.169 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5607049fd930 con 0x560704a86880 2024-08-20T06:43:04.170 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x5607049ece30 con 0x560704a871a0 2024-08-20T06:43:04.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x5607049f0e70 con 0x560704a93130 2024-08-20T06:43:04.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f363bfff640 1 -- 172.21.15.107:0/3426859962 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f36480e37a0 con 0x7f365410ead0 2024-08-20T06:43:04.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 -- 172.21.15.107:0/3423911708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 msgr2=0x560704a95520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a95520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.171 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 -- 172.21.15.107:0/3423911708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 msgr2=0x560704a86c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a86c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.168+0000 7f36527fc640 1 -- 172.21.15.107:0/3423911708 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56070474c650 con 0x560704a871a0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.169+0000 7f36527fc640 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a92bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36480a79d0 tx=0x7f3648010f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f16b08819207f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.169+0000 7f3652ffd640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 0x7f363c0b6190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3644006010 tx=0x7f3644000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.169+0000 7f36397fa640 1 -- 172.21.15.107:0/3423911708 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3648020070 con 0x560704a871a0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.169+0000 7f36397fa640 1 -- 172.21.15.107:0/3423911708 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3648009de0 con 0x560704a871a0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.169+0000 7f36397fa640 1 -- 172.21.15.107:0/3423911708 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36480a69f0 con 0x560704a871a0 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 msgr2=0x560704a92bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a92bf0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f36480a79d0 tx=0x7f3648010f30 comp rx=0 tx=0).stop 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 shutdown_connections 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a95520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a92bf0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.172 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 --2- 172.21.15.107:0/3423911708 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a86c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 >> 172.21.15.107:0/3423911708 conn(0x560704a846b0 msgr2=0x560704a85850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 shutdown_connections 2024-08-20T06:43:04.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- 172.21.15.107:0/3423911708 wait complete. 2024-08-20T06:43:04.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.170+0000 7f365b9f3740 1 -- start start 2024-08-20T06:43:04.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 Processor -- start 2024-08-20T06:43:04.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a93130 0x560704a90780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x56070484f700 con 0x560704a871a0 2024-08-20T06:43:04.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x560704973810 con 0x560704a93130 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x560704a95d20 con 0x560704a86880 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.107:60028/0 (socket says 172.21.15.107:60028) 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f3652ffd640 1 -- 172.21.15.107:0/1673911724 learned_addr learned my addr 172.21.15.107:0/1673911724 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f36527fc640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f36537fe640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a93130 0x560704a90780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.171+0000 7f365b9f3740 10 client.0 _ll_register_callbacks cb 0x5607049e6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 -- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a93130 msgr2=0x560704a90780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a93130 0x560704a90780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 -- 172.21.15.107:0/1673911724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 msgr2=0x560704a8fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 -- 172.21.15.107:0/1673911724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560704a9f050 con 0x560704a86880 2024-08-20T06:43:04.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3652ffd640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3644009900 tx=0x7f3644009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647b28459136e3e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3636ff5640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364404d020 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f3644049480 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56070499c150 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3644035040 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.172+0000 7f365b9f3740 10 client.25663 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560704aa1970 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x56070499c150 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f364403c700 con 0x560704a86880 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 10 client.25663 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 10 client.25663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 20 client.25663 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:04.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704aa2e20 0x560704aa5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 4 client.25663 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:04.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607049ff880 con 0x560704aa2e20 2024-08-20T06:43:04.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.173+0000 7f36527fc640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704aa2e20 0x560704aa5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.174+0000 7f3636ff5640 10 client.25663 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:04.178 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.174+0000 7f36527fc640 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704aa2e20 0x560704aa5260 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3672aeb16c8c0ef5 server_cookie=f43176e066250b03 in_seq=0 out_seq=0 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f3636ff5640 1 -- 172.21.15.107:0/1673911724 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607049ff880 con 0x560704aa2e20 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f3636ff5640 10 client.25663 handle_command_reply: tid=0 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 client.25663 shutdown 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 msgr2=0x560704a92ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3644009900 tx=0x7f3644009260 comp rx=0 tx=0).stop 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 shutdown_connections 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a93130 0x560704a90780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704aa2e20 0x560704aa5260 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 --2- 172.21.15.107:0/1673911724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a86880 0x560704a92ba0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 >> 172.21.15.107:0/1673911724 conn(0x560704a846b0 msgr2=0x560704a8a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f3636ff5640 0 client.25663 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:04.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 shutdown_connections 2024-08-20T06:43:04.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 1 -- 172.21.15.107:0/1673911724 wait complete. 2024-08-20T06:43:04.180 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.175+0000 7f365b9f3740 20 client.25663 trim_cache size 0 max 16384 2024-08-20T06:43:04.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.193+0000 7f365b9f3740 1 Processor -- start 2024-08-20T06:43:04.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.193+0000 7f365b9f3740 1 -- start start 2024-08-20T06:43:04.195 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b6fd0 con 0x560704a871a0 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b48a0 con 0x560704a86880 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f365b9f3740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b26d0 con 0x560704a93130 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f36527fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:51966/0 (socket says 172.21.15.107:51966) 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 -- 172.21.15.107:0/4028277576 learned_addr learned my addr 172.21.15.107:0/4028277576 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 -- 172.21.15.107:0/4028277576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 msgr2=0x560704a8fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f36537fe640 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.196 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 -- 172.21.15.107:0/4028277576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 msgr2=0x560704a8f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f3652ffd640 1 -- 172.21.15.107:0/4028277576 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560704a86fd0 con 0x560704a86880 2024-08-20T06:43:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.194+0000 7f36537fe640 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:43:04.197 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.195+0000 7f3652ffd640 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f364402a930 tx=0x7f3644005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49b77e4534a92128 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.195+0000 7f3638ff9640 1 -- 172.21.15.107:0/4028277576 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364404d040 con 0x560704a86880 2024-08-20T06:43:04.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.195+0000 7f3638ff9640 1 -- 172.21.15.107:0/4028277576 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f364403c8d0 con 0x560704a86880 2024-08-20T06:43:04.199 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.195+0000 7f3638ff9640 1 -- 172.21.15.107:0/4028277576 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644049c20 con 0x560704a86880 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- 172.21.15.107:0/4028277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 msgr2=0x560704a85e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f364402a930 tx=0x7f3644005d50 comp rx=0 tx=0).stop 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- 172.21.15.107:0/4028277576 shutdown_connections 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a871a0 0x560704a8f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 --2- 172.21.15.107:0/4028277576 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a86880 0x560704a85e60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- 172.21.15.107:0/4028277576 >> 172.21.15.107:0/4028277576 conn(0x560704a846b0 msgr2=0x560704a9e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- 172.21.15.107:0/4028277576 shutdown_connections 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- 172.21.15.107:0/4028277576 wait complete. 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.196+0000 7f365b9f3740 1 -- start start 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 Processor -- start 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.200 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a8cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f3652ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:55808/0 (socket says 172.21.15.107:55808) 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f3652ffd640 1 -- 172.21.15.107:0/843636602 learned_addr learned my addr 172.21.15.107:0/843636602 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b2d60 con 0x560704a86880 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b41d0 con 0x560704a871a0 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f36541b3430 con 0x560704a93130 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f36527fc640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a8cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f365b9f3740 10 client.0 _ll_register_callbacks cb 0x7f3644003670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.197+0000 7f36537fe640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 -- 172.21.15.107:0/843636602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 msgr2=0x560704a8d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.201 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 -- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 msgr2=0x560704a8cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a8cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560704a90af0 con 0x560704a86880 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3652ffd640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f364403cc40 tx=0x7f3644049730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b133b034aca7c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3636ff5640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3644069060 con 0x560704a86880 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f364403c650 con 0x560704a86880 2024-08-20T06:43:04.202 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f364403c8d0 con 0x560704a86880 2024-08-20T06:43:04.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f364404d040 con 0x560704a86880 2024-08-20T06:43:04.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f365b9f3740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:04.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.198+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560704b0d880 con 0x560704a86880 2024-08-20T06:43:04.203 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3644049db0 con 0x560704a86880 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f364405b070 con 0x560704a86880 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 20 client.16272 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704b0df70 0x560704b10390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:04.204 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 4 client.16272 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f364403c8d0 con 0x560704b0df70 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.199+0000 7f36527fc640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704b0df70 0x560704b10390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.200+0000 7f3636ff5640 10 client.16272 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.200+0000 7f36527fc640 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704b0df70 0x560704b10390 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e8a397a2f5ebe4d server_cookie=3baad212a8e132ff in_seq=0 out_seq=0 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f3636ff5640 1 -- 172.21.15.107:0/843636602 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f364403c8d0 con 0x560704b0df70 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f3636ff5640 10 client.16272 handle_command_reply: tid=0 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 client.16272 shutdown 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 msgr2=0x560704a9b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f364403cc40 tx=0x7f3644049730 comp rx=0 tx=0).stop 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 shutdown_connections 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x560704b0df70 0x560704b10390 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.205 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560704a93130 0x560704a8d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x560704a871a0 0x560704a8cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f3636ff5640 0 client.16272 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 --2- 172.21.15.107:0/843636602 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x560704a86880 0x560704a9b600 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 >> 172.21.15.107:0/843636602 conn(0x560704a846b0 msgr2=0x560704a82f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 shutdown_connections 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 1 -- 172.21.15.107:0/843636602 wait complete. 2024-08-20T06:43:04.206 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:43:04.207 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.201+0000 7f365b9f3740 20 client.16272 trim_cache size 0 max 16384 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 msgr2=0x7f363c0b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 0x7f363c0b6190 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3644006010 tx=0x7f3644000bc0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 msgr2=0x7f365419faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f364800d950 tx=0x7f364800de10 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 shutdown_connections 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f365410f450 0x7f36541a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f365410ead0 0x7f365419faf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f363c0b3cd0 0x7f363c0b6190 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 --2- 172.21.15.107:0/3426859962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3654109110 0x7f365419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:04.208 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.202+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 >> 172.21.15.107:0/3426859962 conn(0x7f3654078760 msgr2=0x7f3654079310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.203+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 shutdown_connections 2024-08-20T06:43:04.209 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:04.203+0000 7f36595fa640 1 -- 172.21.15.107:0/3426859962 wait complete. 2024-08-20T06:43:05.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:05 smithi107 ceph-mon[18495]: pgmap v514: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-08-20T06:43:05.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:05 smithi183 ceph-mon[22204]: pgmap v514: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-08-20T06:43:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:05 smithi193 ceph-mon[21944]: pgmap v514: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 252 op/s 2024-08-20T06:43:05.887 INFO:tasks.workunit.client.0.smithi107.stdout: 89 11933 133 7897 104 4918 238 2024-08-20T06:43:05.927 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-08-20T06:43:05.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-08-20T06:43:06.267 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:07.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:07 smithi107 ceph-mon[18495]: pgmap v515: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-08-20T06:43:07.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:07 smithi183 ceph-mon[22204]: pgmap v515: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-08-20T06:43:07.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:07 smithi193 ceph-mon[21944]: pgmap v515: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 -- 172.21.15.107:0/2157736366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c108500 msgr2=0x7fd34c108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 --2- 172.21.15.107:0/2157736366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c108500 0x7fd34c108940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009900 tx=0x7fd33c02a410 comp rx=0 tx=0).stop 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 -- 172.21.15.107:0/2157736366 shutdown_connections 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 --2- 172.21.15.107:0/2157736366 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 0x7fd34c10f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 --2- 172.21.15.107:0/2157736366 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c108500 0x7fd34c108940 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 --2- 172.21.15.107:0/2157736366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c102af0 0x7fd34c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 -- 172.21.15.107:0/2157736366 >> 172.21.15.107:0/2157736366 conn(0x7fd34c0fe580 msgr2=0x7fd34c1009a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 -- 172.21.15.107:0/2157736366 shutdown_connections 2024-08-20T06:43:09.255 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.622+0000 7fd353592640 1 -- 172.21.15.107:0/2157736366 wait complete. 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 Processor -- start 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 -- start start 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c108500 0x7fd34c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 0x7fd34c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112980 con 0x7fd34c102af0 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112800 con 0x7fd34c108e80 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd353592640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c112b00 con 0x7fd34c108500 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd35132e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd35132e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52734/0 (socket says 172.21.15.107:52734) 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd35132e640 1 -- 172.21.15.107:0/684729317 learned_addr learned my addr 172.21.15.107:0/684729317 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd350b2d640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c108500 0x7fd34c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.623+0000 7fd351b2f640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 0x7fd34c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.256 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 -- 172.21.15.107:0/684729317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c108500 msgr2=0x7fd34c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c108500 0x7fd34c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 -- 172.21.15.107:0/684729317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 msgr2=0x7fd34c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 0x7fd34c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 -- 172.21.15.107:0/684729317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c1a72d0 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd35132e640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd3427fc640 1 -- 172.21.15.107:0/684729317 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348017030 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd3427fc640 1 -- 172.21.15.107:0/684729317 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd348004020 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c1a75c0 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c1a7b20 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.624+0000 7fd3427fc640 1 -- 172.21.15.107:0/684729317 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480055d0 con 0x7fd34c102af0 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 Processor -- start 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 -- start start 2024-08-20T06:43:09.257 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee991790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee991cd0 0x555fee99d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555fee907940 con 0x555fee99dc60 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555fee8f6e40 con 0x555fee991cd0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35598b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555fee8fae80 con 0x555fee9913b0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd3427fc640 1 -- 172.21.15.107:0/684729317 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fd348009a30 con 0x7fd34c102af0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52742/0 (socket says 172.21.15.107:52742) 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd350b2d640 1 -- 172.21.15.107:0/1470564503 learned_addr learned my addr 172.21.15.107:0/1470564503 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.628+0000 7fd35132e640 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee991cd0 0x555fee99d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd351b2f640 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee991790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 -- 172.21.15.107:0/1470564503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 msgr2=0x555fee991790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.258 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee991790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 -- 172.21.15.107:0/1470564503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee991cd0 msgr2=0x555fee99d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee991cd0 0x555fee99d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 -- 172.21.15.107:0/1470564503 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fee893520 con 0x555fee99dc60 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.629+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd33c02a930 tx=0x7fd33c031ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c96cfa0d0126fa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd3427fc640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 0x7fd3380b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd351b2f640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 0x7fd3380b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd3427fc640 1 -- 172.21.15.107:0/684729317 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd3480e3580 con 0x7fd34c102af0 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd3377fe640 1 -- 172.21.15.107:0/1470564503 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c035070 con 0x555fee99dc60 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd3377fe640 1 -- 172.21.15.107:0/1470564503 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33c040070 con 0x555fee99dc60 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.630+0000 7fd3377fe640 1 -- 172.21.15.107:0/1470564503 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003a80 con 0x555fee99dc60 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd351b2f640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 0x7fd3380b6190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd34c077290 tx=0x7fd344009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- 172.21.15.107:0/1470564503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 msgr2=0x555fee9a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd33c02a930 tx=0x7fd33c031ba0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- 172.21.15.107:0/1470564503 shutdown_connections 2024-08-20T06:43:09.259 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee9a0050 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee991cd0 0x555fee99d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 --2- 172.21.15.107:0/1470564503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee991790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- 172.21.15.107:0/1470564503 >> 172.21.15.107:0/1470564503 conn(0x555fee98f1e0 msgr2=0x555fee990380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- 172.21.15.107:0/1470564503 shutdown_connections 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- 172.21.15.107:0/1470564503 wait complete. 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.631+0000 7fd35598b740 1 -- start start 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 Processor -- start 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee991cd0 0x555fee99b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee99b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x555fee759890 con 0x555fee99dc60 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x555fee87d700 con 0x555fee9913b0 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x555fee9a0950 con 0x555fee991cd0 2024-08-20T06:43:09.260 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35598b740 10 client.0 _ll_register_callbacks cb 0x555fee8f26f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35132e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd351b2f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee99b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35132e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:46486/0 (socket says 172.21.15.107:46486) 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd35132e640 1 -- 172.21.15.107:0/1115444669 learned_addr learned my addr 172.21.15.107:0/1115444669 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.632+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee991cd0 0x555fee99b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 -- 172.21.15.107:0/1115444669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee991cd0 msgr2=0x555fee99b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee991cd0 0x555fee99b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 -- 172.21.15.107:0/1115444669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 msgr2=0x555fee99b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee99b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 -- 172.21.15.107:0/1115444669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fee656930 con 0x555fee9913b0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd351b2f640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee99b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35132e640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd348005fd0 tx=0x7fd3480aa2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35e4f0c73b646ac0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.261 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480aab90 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c040070 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.633+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd3480a6a80 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3480aab90 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 10 client.25799 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fee9ac5f0 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33c040070 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd3480a6720 con 0x555fee9913b0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 10 client.25799 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 10 client.25799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 20 client.25799 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555fee9adeb0 0x555fee9b0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 4 client.25799 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fee909890 con 0x555fee9adeb0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.634+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555fee9adeb0 0x555fee9b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.262 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.635+0000 7fd334ff9640 10 client.25799 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.635+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555fee9adeb0 0x555fee9b0310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad63024bc8deb69 server_cookie=32533f4d25756a2d in_seq=0 out_seq=0 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd334ff9640 1 -- 172.21.15.107:0/1115444669 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555fee909890 con 0x555fee9adeb0 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd334ff9640 10 client.25799 handle_command_reply: tid=0 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 client.25799 shutdown 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 msgr2=0x555fee99acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd348005fd0 tx=0x7fd3480aa2e0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 shutdown_connections 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee99dc60 0x555fee99b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555fee9adeb0 0x555fee9b0310 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee991cd0 0x555fee99b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 --2- 172.21.15.107:0/1115444669 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee9913b0 0x555fee99acc0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd334ff9640 0 client.25799 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:09.263 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 >> 172.21.15.107:0/1115444669 conn(0x555fee98f1e0 msgr2=0x555fee994d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 shutdown_connections 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 1 -- 172.21.15.107:0/1115444669 wait complete. 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.638+0000 7fd35598b740 20 client.25799 trim_cache size 0 max 16384 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 Processor -- start 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 -- start start 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee995420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b2b80 con 0x555fee991cd0 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1b0450 con 0x555fee99dc60 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35598b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ae280 con 0x555fee9913b0 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52752/0 (socket says 172.21.15.107:52752) 2024-08-20T06:43:09.264 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 -- 172.21.15.107:0/4197093522 learned_addr learned my addr 172.21.15.107:0/4197093522 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd351b2f640 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd35132e640 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee995420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 -- 172.21.15.107:0/4197093522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 msgr2=0x555fee995420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee995420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 -- 172.21.15.107:0/4197093522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 msgr2=0x555fee9b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.661+0000 7fd350b2d640 1 -- 172.21.15.107:0/4197093522 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fee98fc60 con 0x555fee991cd0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.662+0000 7fd350b2d640 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009900 tx=0x7fd33c031590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7372138cc0bc09 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.662+0000 7fd336ffd640 1 -- 172.21.15.107:0/4197093522 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c035040 con 0x555fee991cd0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.662+0000 7fd336ffd640 1 -- 172.21.15.107:0/4197093522 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33c040040 con 0x555fee991cd0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.662+0000 7fd336ffd640 1 -- 172.21.15.107:0/4197093522 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003b60 con 0x555fee991cd0 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- 172.21.15.107:0/4197093522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 msgr2=0x555fee9b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.265 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd33c009900 tx=0x7fd33c031590 comp rx=0 tx=0).stop 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- 172.21.15.107:0/4197093522 shutdown_connections 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9b5c20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 --2- 172.21.15.107:0/4197093522 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee995420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- 172.21.15.107:0/4197093522 >> 172.21.15.107:0/4197093522 conn(0x555fee98f1e0 msgr2=0x555fee99cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- 172.21.15.107:0/4197093522 shutdown_connections 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- 172.21.15.107:0/4197093522 wait complete. 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 -- start start 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.663+0000 7fd35598b740 1 Processor -- start 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee9a6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9a1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1ae910 con 0x555fee991cd0 2024-08-20T06:43:09.266 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1afd80 con 0x555fee99dc60 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c1aefe0 con 0x555fee9913b0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35598b740 10 client.0 _ll_register_callbacks cb 0x7fd3480a6350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd350b2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52766/0 (socket says 172.21.15.107:52766) 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee9a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd350b2d640 1 -- 172.21.15.107:0/1319228374 learned_addr learned my addr 172.21.15.107:0/1319228374 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd351b2f640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9a1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 -- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 msgr2=0x555fee9a1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9a1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 -- 172.21.15.107:0/1319228374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 msgr2=0x555fee9a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd35132e640 1 -- 172.21.15.107:0/1319228374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fee9a20c0 con 0x555fee9913b0 2024-08-20T06:43:09.267 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.664+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35132e640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee9a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3480e5780 tx=0x7fd3480ae120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54c2acba1edec9a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 10 client.0 ms_handle_connect on v2:172.21.15.193:3300/0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480b3540 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd33c040040 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mon.1 v2:172.21.15.193:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd3480a8860 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3480aa490 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 10 client.25684 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555feea18a80 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mon.1 v2:172.21.15.193:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd33c040040 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mon.1 v2:172.21.15.193:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd3480a6720 con 0x555fee9913b0 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 10 client.25684 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 10 client.25684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 20 client.25684 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.665+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555feea19270 0x555feea1b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:09.268 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.666+0000 7fd35598b740 4 client.25684 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.666+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555fee905330 con 0x555feea19270 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.666+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555feea19270 0x555feea1b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.666+0000 7fd334ff9640 10 client.25684 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.666+0000 7fd350b2d640 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555feea19270 0x555feea1b690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f91bc83db9836d95 server_cookie=7de1b78ee42f05b4 in_seq=0 out_seq=0 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd334ff9640 1 -- 172.21.15.107:0/1319228374 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555fee905330 con 0x555feea19270 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd334ff9640 10 client.25684 handle_command_reply: tid=0 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 client.25684 shutdown 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 msgr2=0x555fee9a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee9a6070 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd3480e5780 tx=0x7fd3480ae120 comp rx=0 tx=0).stop 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:09.269 INFO:teuthology.orchestra.run.smithi107.stdout: "return_code": 0, 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "scrub_tag": "b3c94ad0-e4c6-45d1-b2bd-3cd5b75fa4c4", 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": "asynchronous" 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 shutdown_connections 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x555fee99dc60 0x555fee9a1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x555fee991cd0 0x555fee9a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x555feea19270 0x555feea1b690 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 --2- 172.21.15.107:0/1319228374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555fee9913b0 0x555fee9a6070 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 >> 172.21.15.107:0/1319228374 conn(0x555fee98f1e0 msgr2=0x555fee98dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd334ff9640 0 client.25684 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 shutdown_connections 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 1 -- 172.21.15.107:0/1319228374 wait complete. 2024-08-20T06:43:09.270 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.667+0000 7fd35598b740 20 client.25684 trim_cache size 0 max 16384 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 msgr2=0x7fd3380b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 0x7fd3380b6190 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd34c077290 tx=0x7fd344009500 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 msgr2=0x7fd34c0751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fd34800d950 tx=0x7fd34800de10 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 shutdown_connections 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd34c108e80 0x7fd34c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd3380b3cd0 0x7fd3380b6190 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd34c108500 0x7fd34c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 --2- 172.21.15.107:0/684729317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd34c102af0 0x7fd34c0751d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 >> 172.21.15.107:0/684729317 conn(0x7fd34c0fe580 msgr2=0x7fd34c100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 shutdown_connections 2024-08-20T06:43:09.271 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:08.668+0000 7fd353592640 1 -- 172.21.15.107:0/684729317 wait complete. 2024-08-20T06:43:09.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:09 smithi183 ceph-mon[22204]: pgmap v516: 97 pgs: 97 active+clean; 399 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-08-20T06:43:09.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:09 smithi193 ceph-mon[21944]: pgmap v516: 97 pgs: 97 active+clean; 399 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-08-20T06:43:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:09 smithi107 ceph-mon[18495]: pgmap v516: 97 pgs: 97 active+clean; 399 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 240 op/s 2024-08-20T06:43:09.999 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:43:10.146 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-08-20T06:43:10.146 INFO:tasks.workunit.client.0.smithi107.stdout:Final score for writes: 89 2024-08-20T06:43:10.146 INFO:tasks.workunit.client.0.smithi107.stdout:Final score for reads : 8176 2024-08-20T06:43:10.146 INFO:tasks.workunit.client.0.smithi107.stdout: 2024-08-20T06:43:10.147 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-08-20T06:43:10.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-08-20T06:43:10.290 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub queued for path: / 2024-08-20T06:43:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: active paths [/] 2024-08-20T06:43:10.363 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:10 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub queued for path: / 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: active paths [/] 2024-08-20T06:43:10.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:10 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub queued for path: / 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: active paths [/] 2024-08-20T06:43:10.557 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:10 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:10.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2439618512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0107d20 msgr2=0x7fd2f010e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2439618512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0107d20 0x7fd2f010e520 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009ca0 tx=0x7fd2e802a8c0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2439618512 shutdown_connections 2024-08-20T06:43:10.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2439618512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f0107d20 0x7fd2f010e520 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.855 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2439618512 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f01073a0 0x7fd2f01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2439618512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 0x7fd2f0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2439618512 >> 172.21.15.107:0/2439618512 conn(0x7fd2f00fd400 msgr2=0x7fd2f00ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2439618512 shutdown_connections 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2439618512 wait complete. 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 Processor -- start 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.819+0000 7fd2f6ef7640 1 -- start start 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 0x7fd2f0198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.856 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f0107d20 0x7fd2f019e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01129e0 con 0x7fd2f01073a0 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0112860 con 0x7fd2f0107d20 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f0112b60 con 0x7fd2f0101990 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52796/0 (socket says 172.21.15.107:52796) 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 -- 172.21.15.107:0/2607588524 learned_addr learned my addr 172.21.15.107:0/2607588524 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 0x7fd2f0198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f5494640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f0107d20 0x7fd2f019e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 -- 172.21.15.107:0/2607588524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 msgr2=0x7fd2f0198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 0x7fd2f0198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 -- 172.21.15.107:0/2607588524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f0107d20 msgr2=0x7fd2f019e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.857 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f0107d20 0x7fd2f019e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 -- 172.21.15.107:0/2607588524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f019ef40 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2effff640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00e9f0 tx=0x7fd2dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2edffb640 1 -- 172.21.15.107:0/2607588524 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc00cd80 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2edffb640 1 -- 172.21.15.107:0/2607588524 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2dc0108c0 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f019f230 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.820+0000 7fd2edffb640 1 -- 172.21.15.107:0/2607588524 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0187c0 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.821+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f01a58f0 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2edffb640 1 -- 172.21.15.107:0/2607588524 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fd2dc010430 con 0x7fd2f01073a0 2024-08-20T06:43:10.858 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 Processor -- start 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 -- start start 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40996ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d409970e0 0x556d409994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556d409018e0 con 0x556d4098a830 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d408f0de0 con 0x556d409970e0 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.823+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d408f4e20 con 0x556d4098b150 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.859 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52802/0 (socket says 172.21.15.107:52802) 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 -- 172.21.15.107:0/3937055719 learned_addr learned my addr 172.21.15.107:0/3937055719 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2effff640 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d409970e0 0x556d409994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40996ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 -- 172.21.15.107:0/3937055719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 msgr2=0x556d40996ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40996ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 -- 172.21.15.107:0/3937055719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d409970e0 msgr2=0x556d409994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d409970e0 0x556d409994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 -- 172.21.15.107:0/3937055719 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d40650650 con 0x556d4098a830 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2edffb640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 0x7fd2e00b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.860 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f5494640 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8035000 tx=0x7fd2e80093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f37f29b27521520 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 0x7fd2e00b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2edffb640 1 -- 172.21.15.107:0/2607588524 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd2dc0e3f70 con 0x7fd2f01073a0 2024-08-20T06:43:10.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.824+0000 7fd2daffd640 1 -- 172.21.15.107:0/3937055719 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8031980 con 0x556d4098a830 2024-08-20T06:43:10.861 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2daffd640 1 -- 172.21.15.107:0/3937055719 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2e8036070 con 0x556d4098a830 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2daffd640 1 -- 172.21.15.107:0/3937055719 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004420 con 0x556d4098a830 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 0x7fd2e00b6090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x556d4098d9b0 tx=0x7fd2e400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3937055719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 msgr2=0x556d4098ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8035000 tx=0x7fd2e80093f0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3937055719 shutdown_connections 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d409970e0 0x556d409994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40996ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3937055719 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098a830 0x556d4098ac10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3937055719 >> 172.21.15.107:0/3937055719 conn(0x556d40988660 msgr2=0x556d40989800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.862 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3937055719 shutdown_connections 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.825+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3937055719 wait complete. 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 -- start start 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 Processor -- start 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d40994100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40994b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x556d40753700 con 0x556d4098b150 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x556d408777c0 con 0x556d4098a830 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x556d40999bb0 con 0x556d409970e0 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f92f0740 10 client.0 _ll_register_callbacks cb 0x556d408ea6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.863 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52806/0 (socket says 172.21.15.107:52806) 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2effff640 1 -- 172.21.15.107:0/3743353181 learned_addr learned my addr 172.21.15.107:0/3743353181 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d40994100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.826+0000 7fd2f5494640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40994b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 -- 172.21.15.107:0/3743353181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 msgr2=0x556d40994b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40994b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 -- 172.21.15.107:0/3743353181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 msgr2=0x556d40994640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 -- 172.21.15.107:0/3743353181 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d409a2fe0 con 0x556d4098a830 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2effff640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d40994100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4004650 tx=0x7fd2e4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80d83e4ae66470bd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2cffff640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e401fc90 con 0x556d4098a830 2024-08-20T06:43:10.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d408a0100 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd2e4017030 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2e401fe30 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f92f0740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d409a5a40 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.827+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x556d408a0100 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd2e4004ca0 con 0x556d4098a830 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 10 client.25817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 20 client.25817 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d409a6ef0 0x556d409a9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 4 client.25817 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:10.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d40903830 con 0x556d409a6ef0 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2effff640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d409a6ef0 0x556d409a9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2cffff640 10 client.25817 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.828+0000 7fd2effff640 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d409a6ef0 0x556d409a9370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78b8505f36a55ca server_cookie=3cac8b45d526592 in_seq=0 out_seq=0 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2cffff640 1 -- 172.21.15.107:0/3743353181 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d40903830 con 0x556d409a6ef0 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2cffff640 10 client.25817 handle_command_reply: tid=0 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2f92f0740 1 client.25817 shutdown 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 msgr2=0x556d40994100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d40994100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4004650 tx=0x7fd2e4004730 comp rx=0 tx=0).stop 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.829+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 shutdown_connections 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d409a6ef0 0x556d409a9370 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40994b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d40994640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2cffff640 0 client.25817 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/3743353181 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d40994100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 >> 172.21.15.107:0/3743353181 conn(0x556d40988660 msgr2=0x556d4098e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 shutdown_connections 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 1 -- 172.21.15.107:0/3743353181 wait complete. 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.830+0000 7fd2f92f0740 20 client.25817 trim_cache size 0 max 16384 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 Processor -- start 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 -- start start 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d4098e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40993990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01b0880 con 0x556d409970e0 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01ae150 con 0x556d4098a830 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01abf80 con 0x556d4098b150 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.849+0000 7fd2f5494640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52810/0 (socket says 172.21.15.107:52810) 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 -- 172.21.15.107:0/1087097049 learned_addr learned my addr 172.21.15.107:0/1087097049 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d4098e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2effff640 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40993990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 -- 172.21.15.107:0/1087097049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 msgr2=0x556d40993990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40993990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 -- 172.21.15.107:0/1087097049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 msgr2=0x556d4098e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d4098e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 -- 172.21.15.107:0/1087097049 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d40988490 con 0x556d409970e0 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2f5494640 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005fe0 tx=0x7fd2e80040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a59a5281d81e4f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2da7fc640 1 -- 172.21.15.107:0/1087097049 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8053070 con 0x556d409970e0 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2da7fc640 1 -- 172.21.15.107:0/1087097049 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2e804b070 con 0x556d409970e0 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.850+0000 7fd2da7fc640 1 -- 172.21.15.107:0/1087097049 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8004150 con 0x556d409970e0 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- 172.21.15.107:0/1087097049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 msgr2=0x556d40987610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005fe0 tx=0x7fd2e80040e0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- 172.21.15.107:0/1087097049 shutdown_connections 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d409970e0 0x556d40987610 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d4098b150 0x556d40993990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.870 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/1087097049 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d4098e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- 172.21.15.107:0/1087097049 >> 172.21.15.107:0/1087097049 conn(0x556d40988660 msgr2=0x556d40986f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- 172.21.15.107:0/1087097049 shutdown_connections 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- 172.21.15.107:0/1087097049 wait complete. 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 -- start start 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.851+0000 7fd2f92f0740 1 Processor -- start 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d409ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40a0dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.871 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01ac610 con 0x556d4098b150 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01ada80 con 0x556d4098a830 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f01acce0 con 0x556d409970e0 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f92f0740 10 client.0 _ll_register_callbacks cb 0x7fd2e4005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f4c93640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d409ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52816/0 (socket says 172.21.15.107:52816) 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 -- 172.21.15.107:0/2601267356 learned_addr learned my addr 172.21.15.107:0/2601267356 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f5494640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40a0dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 -- 172.21.15.107:0/2601267356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 msgr2=0x556d40a0dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40a0dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 -- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 msgr2=0x556d409ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d409ae3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2effff640 1 -- 172.21.15.107:0/2601267356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d409890e0 con 0x556d4098b150 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f5494640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40a0dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.852+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d409ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2effff640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0e3010 tx=0x7fd2dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9341eed4ce486b2a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc0be020 con 0x556d4098b150 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fd2dc0e1b80 con 0x556d4098b150 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2e804b070 con 0x556d4098b150 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2dc0aaa80 con 0x556d4098b150 2024-08-20T06:43:10.873 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 10 client.16308 fetch_fsmap learned FSMap version 29 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d40a155c0 con 0x556d4098b150 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fd2e804b070 con 0x556d4098b150 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd2dc0e7040 con 0x556d4098b150 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 10 client.16308 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 20 client.16308 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d40a15dd0 0x556d40a181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 4 client.16308 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.853+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d408ff2d0 con 0x556d40a15dd0 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.854+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d40a15dd0 0x556d40a181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.854+0000 7fd2f4c93640 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d40a15dd0 0x556d40a181f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f80d4ed8833b8c4 server_cookie=61a26e8ed7888d15 in_seq=0 out_seq=0 2024-08-20T06:43:10.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.854+0000 7fd2cffff640 10 client.16308 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:10.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.855+0000 7fd2cffff640 1 -- 172.21.15.107:0/2601267356 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x556d408ff2d0 con 0x556d40a15dd0 2024-08-20T06:43:10.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.855+0000 7fd2cffff640 10 client.16308 handle_command_reply: tid=0 2024-08-20T06:43:10.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.855+0000 7fd2f92f0740 1 client.16308 shutdown 2024-08-20T06:43:10.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 msgr2=0x556d409ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc0e3010 tx=0x7fd2dc009310 comp rx=0 tx=0).stop 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 shutdown_connections 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x556d40a15dd0 0x556d40a181f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d409970e0 0x556d40a0dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x556d4098b150 0x556d409ae980 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 --2- 172.21.15.107:0/2601267356 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x556d4098a830 0x556d409ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 >> 172.21.15.107:0/2601267356 conn(0x556d40988660 msgr2=0x556d40987720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2cffff640 0 client.16308 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 shutdown_connections 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 1 -- 172.21.15.107:0/2601267356 wait complete. 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f92f0740 20 client.16308 trim_cache size 0 max 16384 2024-08-20T06:43:10.877 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "scrub active (7283 inodes in the stack)", 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": { 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "b3c94ad0-e4c6-45d1-b2bd-3cd5b75fa4c4": { 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "path": "/", 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tag": "b3c94ad0-e4c6-45d1-b2bd-3cd5b75fa4c4", 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: "options": "recursive,force" 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:10.878 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:10.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 msgr2=0x7fd2e00b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 0x7fd2e00b6090 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x556d4098d9b0 tx=0x7fd2e400b390 comp rx=0 tx=0).stop 2024-08-20T06:43:10.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 msgr2=0x7fd2f01993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:10.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.856+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00e9f0 tx=0x7fd2dc00eeb0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 shutdown_connections 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fd2f0107d20 0x7fd2f019e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd2f01073a0 0x7fd2f01993d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fd2e00b3bd0 0x7fd2e00b6090 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 --2- 172.21.15.107:0/2607588524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2f0101990 0x7fd2f0198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 >> 172.21.15.107:0/2607588524 conn(0x7fd2f00fd400 msgr2=0x7fd2f00ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 shutdown_connections 2024-08-20T06:43:10.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:10.857+0000 7fd2f6ef7640 1 -- 172.21.15.107:0/2607588524 wait complete. 2024-08-20T06:43:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:11 smithi183 ceph-mon[22204]: pgmap v517: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-08-20T06:43:11.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:11 smithi193 ceph-mon[21944]: pgmap v517: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-08-20T06:43:11.578 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:11 smithi107 ceph-mon[18495]: pgmap v517: 97 pgs: 97 active+clean; 402 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 239 op/s 2024-08-20T06:43:11.677 INFO:tasks.cephfs.filesystem:scrub status for tag:b3c94ad0-e4c6-45d1-b2bd-3cd5b75fa4c4 - {'path': '/', 'tag': 'b3c94ad0-e4c6-45d1-b2bd-3cd5b75fa4c4', 'options': 'recursive,force'} 2024-08-20T06:43:13.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:13 smithi183 ceph-mon[22204]: pgmap v518: 97 pgs: 97 active+clean; 407 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 346 op/s 2024-08-20T06:43:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:13 smithi193 ceph-mon[21944]: pgmap v518: 97 pgs: 97 active+clean; 407 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 346 op/s 2024-08-20T06:43:13.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:13 smithi107 ceph-mon[18495]: pgmap v518: 97 pgs: 97 active+clean; 407 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 346 op/s 2024-08-20T06:43:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:43:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:43:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:43:14.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:43:14.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:43:14.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:43:14.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:43:14.512 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:43:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:43:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:43:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:43:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:43:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:15 smithi183 ceph-mon[22204]: pgmap v519: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 768 op/s 2024-08-20T06:43:15.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:15 smithi183 ceph-mon[22204]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:15 smithi193 ceph-mon[21944]: pgmap v519: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 768 op/s 2024-08-20T06:43:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:15 smithi193 ceph-mon[21944]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:15 smithi107 ceph-mon[18495]: pgmap v519: 97 pgs: 97 active+clean; 405 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 768 op/s 2024-08-20T06:43:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:15 smithi107 ceph-mon[18495]: scrub summary: idle+waiting paths [/] 2024-08-20T06:43:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:16 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:43:16.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:16 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:43:16.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:16 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:43:16.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:16 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:43:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:16 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:43:16.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:16 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:43:17.387 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:17 smithi183 ceph-mon[22204]: scrub summary: idle 2024-08-20T06:43:17.387 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:17 smithi183 ceph-mon[22204]: pgmap v520: 97 pgs: 97 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 894 op/s 2024-08-20T06:43:17.455 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:17 smithi193 ceph-mon[21944]: scrub summary: idle 2024-08-20T06:43:17.456 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:17 smithi193 ceph-mon[21944]: pgmap v520: 97 pgs: 97 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 894 op/s 2024-08-20T06:43:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:17 smithi107 ceph-mon[18495]: scrub summary: idle 2024-08-20T06:43:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:17 smithi107 ceph-mon[18495]: pgmap v520: 97 pgs: 97 active+clean; 406 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 894 op/s 2024-08-20T06:43:18.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:18 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:18.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:18 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:18.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:18 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:19.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:19 smithi183 ceph-mon[22204]: pgmap v521: 97 pgs: 97 active+clean; 390 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.06k op/s 2024-08-20T06:43:19.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:19 smithi193 ceph-mon[21944]: pgmap v521: 97 pgs: 97 active+clean; 390 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.06k op/s 2024-08-20T06:43:19.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:19 smithi107 ceph-mon[18495]: pgmap v521: 97 pgs: 97 active+clean; 390 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 1.06k op/s 2024-08-20T06:43:21.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:21 smithi183 ceph-mon[22204]: pgmap v522: 97 pgs: 97 active+clean; 370 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 1.12k op/s 2024-08-20T06:43:21.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:21 smithi193 ceph-mon[21944]: pgmap v522: 97 pgs: 97 active+clean; 370 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 1.12k op/s 2024-08-20T06:43:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:21 smithi107 ceph-mon[18495]: pgmap v522: 97 pgs: 97 active+clean; 370 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 1.12k op/s 2024-08-20T06:43:23.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:23 smithi183 ceph-mon[22204]: pgmap v523: 97 pgs: 97 active+clean; 364 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.15k op/s 2024-08-20T06:43:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:23 smithi193 ceph-mon[21944]: pgmap v523: 97 pgs: 97 active+clean; 364 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.15k op/s 2024-08-20T06:43:23.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:23 smithi107 ceph-mon[18495]: pgmap v523: 97 pgs: 97 active+clean; 364 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.15k op/s 2024-08-20T06:43:25.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:25 smithi183 ceph-mon[22204]: pgmap v524: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.13k op/s 2024-08-20T06:43:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:25 smithi193 ceph-mon[21944]: pgmap v524: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.13k op/s 2024-08-20T06:43:25.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:25 smithi107 ceph-mon[18495]: pgmap v524: 97 pgs: 97 active+clean; 346 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 1.13k op/s 2024-08-20T06:43:27.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:27 smithi183 ceph-mon[22204]: pgmap v525: 97 pgs: 97 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 706 op/s 2024-08-20T06:43:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:27 smithi193 ceph-mon[21944]: pgmap v525: 97 pgs: 97 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 706 op/s 2024-08-20T06:43:27.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:27 smithi107 ceph-mon[18495]: pgmap v525: 97 pgs: 97 active+clean; 341 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 706 op/s 2024-08-20T06:43:29.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:29 smithi183 ceph-mon[22204]: pgmap v526: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 606 op/s 2024-08-20T06:43:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:29 smithi193 ceph-mon[21944]: pgmap v526: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 606 op/s 2024-08-20T06:43:29.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:29 smithi107 ceph-mon[18495]: pgmap v526: 97 pgs: 97 active+clean; 326 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 606 op/s 2024-08-20T06:43:31.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:31 smithi183 ceph-mon[22204]: pgmap v527: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2024-08-20T06:43:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:31 smithi193 ceph-mon[21944]: pgmap v527: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2024-08-20T06:43:31.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:31 smithi107 ceph-mon[18495]: pgmap v527: 97 pgs: 97 active+clean; 314 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 486 op/s 2024-08-20T06:43:33.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:33 smithi183 ceph-mon[22204]: pgmap v528: 97 pgs: 97 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 446 op/s 2024-08-20T06:43:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:33 smithi193 ceph-mon[21944]: pgmap v528: 97 pgs: 97 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 446 op/s 2024-08-20T06:43:33.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:33 smithi107 ceph-mon[18495]: pgmap v528: 97 pgs: 97 active+clean; 306 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 446 op/s 2024-08-20T06:43:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:35 smithi183 ceph-mon[22204]: pgmap v529: 97 pgs: 97 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 489 op/s 2024-08-20T06:43:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:35 smithi193 ceph-mon[21944]: pgmap v529: 97 pgs: 97 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 489 op/s 2024-08-20T06:43:35.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:35 smithi107 ceph-mon[18495]: pgmap v529: 97 pgs: 97 active+clean; 277 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 489 op/s 2024-08-20T06:43:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:37 smithi183 ceph-mon[22204]: pgmap v530: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-08-20T06:43:37.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:37 smithi193 ceph-mon[21944]: pgmap v530: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-08-20T06:43:37.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:37 smithi107 ceph-mon[18495]: pgmap v530: 97 pgs: 97 active+clean; 266 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 1.0 MiB/s wr, 409 op/s 2024-08-20T06:43:39.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:39 smithi183 ceph-mon[22204]: pgmap v531: 97 pgs: 97 active+clean; 256 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-08-20T06:43:39.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:39 smithi193 ceph-mon[21944]: pgmap v531: 97 pgs: 97 active+clean; 256 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-08-20T06:43:39.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:39 smithi107 ceph-mon[18495]: pgmap v531: 97 pgs: 97 active+clean; 256 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 444 op/s 2024-08-20T06:43:40.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:40 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:40.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:40 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:40.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:40 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:43:41.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:41 smithi183 ceph-mon[22204]: pgmap v532: 97 pgs: 97 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-08-20T06:43:41.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:41 smithi193 ceph-mon[21944]: pgmap v532: 97 pgs: 97 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-08-20T06:43:41.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:41 smithi107 ceph-mon[18495]: pgmap v532: 97 pgs: 97 active+clean; 246 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 417 op/s 2024-08-20T06:43:41.679 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-08-20T06:43:41.939 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 -- 172.21.15.107:0/708731342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1019a0 msgr2=0x7f8bfc101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 --2- 172.21.15.107:0/708731342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc101d80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf802a410 comp rx=0 tx=0).stop 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 -- 172.21.15.107:0/708731342 shutdown_connections 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 --2- 172.21.15.107:0/708731342 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc107d30 0x7f8bfc10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 --2- 172.21.15.107:0/708731342 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 --2- 172.21.15.107:0/708731342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc101d80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 -- 172.21.15.107:0/708731342 >> 172.21.15.107:0/708731342 conn(0x7f8bfc0fd410 msgr2=0x7f8bfc0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 -- 172.21.15.107:0/708731342 shutdown_connections 2024-08-20T06:43:42.675 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.651+0000 7f8c03876640 1 -- 172.21.15.107:0/708731342 wait complete. 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 Processor -- start 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 -- start start 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc107d30 0x7f8bfc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1118a0 con 0x7f8bfc1073b0 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc111720 con 0x7f8bfc1019a0 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.652+0000 7f8c03876640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc111a20 con 0x7f8bfc107d30 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40142/0 (socket says 172.21.15.107:40142) 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 -- 172.21.15.107:0/1096113550 learned_addr learned my addr 172.21.15.107:0/1096113550 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c01612640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.676 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c01e13640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc107d30 0x7f8bfc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 -- 172.21.15.107:0/1096113550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc107d30 msgr2=0x7f8bfc19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc107d30 0x7f8bfc19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 -- 172.21.15.107:0/1096113550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc1019a0 msgr2=0x7f8bfc198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 -- 172.21.15.107:0/1096113550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bfc19ef70 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c00e11640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e9f0 tx=0x7f8bec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8bf27fc640 1 -- 172.21.15.107:0/1096113550 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec00cd80 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8bf27fc640 1 -- 172.21.15.107:0/1096113550 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bec0054b0 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bfc19f260 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8bf27fc640 1 -- 172.21.15.107:0/1096113550 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec010430 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.653+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bfc1a5920 con 0x7f8bfc1073b0 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 Processor -- start 2024-08-20T06:43:42.677 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8bf27fc640 1 -- 172.21.15.107:0/1096113550 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f8bec0046c0 con 0x7f8bfc1073b0 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 -- start start 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 0x55bdb383cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb38490e0 0x55bdb384b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37b38e0 con 0x55bdb38490e0 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37a2de0 con 0x55bdb383d150 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37a6e20 con 0x55bdb383c830 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb38490e0 0x55bdb384b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.656+0000 7f8c01612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:59854/0 (socket says 172.21.15.107:59854) 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 -- 172.21.15.107:0/1920292330 learned_addr learned my addr 172.21.15.107:0/1920292330 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:42.678 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01e13640 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 0x55bdb383cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 -- 172.21.15.107:0/1920292330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 msgr2=0x55bdb383cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 0x55bdb383cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 -- 172.21.15.107:0/1920292330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb38490e0 msgr2=0x55bdb384b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb38490e0 0x55bdb384b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 -- 172.21.15.107:0/1920292330 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb3502650 con 0x55bdb383d150 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01e13640 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 0x55bdb383cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8bf27fc640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 0x7f8be80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01612640 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8000c00 tx=0x7f8bf8031a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=742a39d62fcf3ae8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8c01e13640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 0x7f8be80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8bf27fc640 1 -- 172.21.15.107:0/1096113550 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8bec0e4010 con 0x7f8bfc1073b0 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8be77fe640 1 -- 172.21.15.107:0/1920292330 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8035030 con 0x55bdb383d150 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8be77fe640 1 -- 172.21.15.107:0/1920292330 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bf8040070 con 0x55bdb383d150 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.657+0000 7f8be77fe640 1 -- 172.21.15.107:0/1920292330 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8003850 con 0x55bdb383d150 2024-08-20T06:43:42.679 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1920292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 msgr2=0x55bdb3848ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c01e13640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 0x7f8be80b6090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005fd0 tx=0x7f8bf4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8000c00 tx=0x7f8bf8031a70 comp rx=0 tx=0).stop 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1920292330 shutdown_connections 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb38490e0 0x55bdb384b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3848ba0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1920292330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb383c830 0x55bdb383cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1920292330 >> 172.21.15.107:0/1920292330 conn(0x55bdb383a660 msgr2=0x55bdb383b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1920292330 shutdown_connections 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1920292330 wait complete. 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.658+0000 7f8c05c6f740 1 -- start start 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 Processor -- start 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb3846060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.680 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3846ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55bdb3605700 con 0x55bdb383d150 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55bdb37297c0 con 0x55bdb383c830 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55bdb384bd30 con 0x55bdb38490e0 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c05c6f740 10 client.0 _ll_register_callbacks cb 0x55bdb379c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40152/0 (socket says 172.21.15.107:40152) 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c01612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb3846060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 -- 172.21.15.107:0/2972592478 learned_addr learned my addr 172.21.15.107:0/2972592478 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c01e13640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3846ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 -- 172.21.15.107:0/2972592478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 msgr2=0x55bdb3846ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.681 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3846ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 -- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 msgr2=0x55bdb3846060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb3846060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c00e11640 1 -- 172.21.15.107:0/2972592478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb3855140 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.659+0000 7f8c01612640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb3846060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0a6fe0 tx=0x7f8bec005660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743a63fe7a0de288 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0aa8b0 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8bec0e7c00 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bdb3752100 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bec0aa6d0 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 10 client.16320 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bdb3857b30 con 0x55bdb383d150 2024-08-20T06:43:42.682 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55bdb3752100 con 0x55bdb383d150 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bec0ae070 con 0x55bdb383d150 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 10 client.16320 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 20 client.16320 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.660+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb3859060 0x55bdb385b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.661+0000 7f8c05c6f740 4 client.16320 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.661+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdb37b5830 con 0x55bdb3859060 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.661+0000 7f8c01612640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb3859060 0x55bdb385b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.661+0000 7f8be4ff9640 10 client.16320 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.661+0000 7f8c01612640 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb3859060 0x55bdb385b470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6c570b2bdc0a35a server_cookie=186611e748aafe19 in_seq=0 out_seq=0 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.662+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2972592478 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdb37b5830 con 0x55bdb3859060 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.662+0000 7f8be4ff9640 10 client.16320 handle_command_reply: tid=0 2024-08-20T06:43:42.683 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.662+0000 7f8c05c6f740 1 client.16320 shutdown 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 msgr2=0x55bdb38465a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0a6fe0 tx=0x7f8bec005660 comp rx=0 tx=0).stop 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 shutdown_connections 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb3859060 0x55bdb385b470 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3846ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb38465a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2972592478 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb3846060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 >> 172.21.15.107:0/2972592478 conn(0x55bdb383a660 msgr2=0x55bdb3840200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8be4ff9640 0 client.16320 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 shutdown_connections 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2972592478 wait complete. 2024-08-20T06:43:42.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.663+0000 7f8c05c6f740 20 client.16320 trim_cache size 0 max 16384 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 Processor -- start 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 -- start start 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb3845910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3860d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1b08e0 con 0x55bdb383d150 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1ae150 con 0x55bdb383c830 2024-08-20T06:43:42.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1abf80 con 0x55bdb38490e0 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c01612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb3845910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c01612640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:59876/0 (socket says 172.21.15.107:59876) 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.681+0000 7f8c01612640 1 -- 172.21.15.107:0/1560970975 learned_addr learned my addr 172.21.15.107:0/1560970975 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 -- 172.21.15.107:0/1560970975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 msgr2=0x55bdb3860d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3860d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 -- 172.21.15.107:0/1560970975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 msgr2=0x55bdb3845910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb3845910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 -- 172.21.15.107:0/1560970975 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdb383cf80 con 0x55bdb383c830 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8c01612640 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55bdb383f9b0 tx=0x7f8bf802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a571023d26d29386 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8be6ffd640 1 -- 172.21.15.107:0/1560970975 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8035040 con 0x55bdb383c830 2024-08-20T06:43:42.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8be6ffd640 1 -- 172.21.15.107:0/1560970975 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bf8040040 con 0x55bdb383c830 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.682+0000 7f8be6ffd640 1 -- 172.21.15.107:0/1560970975 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8058460 con 0x55bdb383c830 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1560970975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 msgr2=0x55bdb38409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x55bdb383f9b0 tx=0x7f8bf802a410 comp rx=0 tx=0).stop 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1560970975 shutdown_connections 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3860d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383d150 0x55bdb3845910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/1560970975 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383c830 0x55bdb38409f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1560970975 >> 172.21.15.107:0/1560970975 conn(0x55bdb383a660 msgr2=0x55bdb3847eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1560970975 shutdown_connections 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- 172.21.15.107:0/1560970975 wait complete. 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 -- start start 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.683+0000 7f8c05c6f740 1 Processor -- start 2024-08-20T06:43:42.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383c830 0x55bdb3851020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3842370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1ac610 con 0x55bdb383c830 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1ada80 con 0x55bdb383d150 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8bfc1acce0 con 0x55bdb38490e0 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c05c6f740 10 client.0 _ll_register_callbacks cb 0x7f8bec0e7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:59890/0 (socket says 172.21.15.107:59890) 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 -- 172.21.15.107:0/2556174725 learned_addr learned my addr 172.21.15.107:0/2556174725 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c01612640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383c830 0x55bdb3851020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c01e13640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3842370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 -- 172.21.15.107:0/2556174725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 msgr2=0x55bdb3842370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3842370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 -- 172.21.15.107:0/2556174725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383c830 msgr2=0x55bdb3851020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383c830 0x55bdb3851020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.684+0000 7f8c00e11640 1 -- 172.21.15.107:0/2556174725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdb3851c00 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c00e11640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0b41d0 tx=0x7f8bec005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63b4d7765c8958a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.183:3300/0 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0aa8b0 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mon.2 v2:172.21.15.183:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8bec0b3b80 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bf8040040 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bec0a6e20 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 10 client.25841 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bdb38c3330 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mon.2 v2:172.21.15.183:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8bf8040040 con 0x55bdb383d150 2024-08-20T06:43:42.693 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mon.2 v2:172.21.15.183:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bec0ae040 con 0x55bdb383d150 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 10 client.25841 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 10 client.25841 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 20 client.25841 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb38c3b90 0x55bdb38c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 4 client.25841 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.685+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bdb37b12d0 con 0x55bdb38c3b90 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c01612640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb38c3b90 0x55bdb38c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8be4ff9640 10 client.25841 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c01612640 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb38c3b90 0x55bdb38c5fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49f4b157177c1a9b server_cookie=61d49df3144aadd6 in_seq=0 out_seq=0 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8be4ff9640 1 -- 172.21.15.107:0/2556174725 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bdb37b12d0 con 0x55bdb38c3b90 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8be4ff9640 10 client.25841 handle_command_reply: tid=0 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 client.25841 shutdown 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 msgr2=0x55bdb3851560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.694 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0b41d0 tx=0x7f8bec005b20 comp rx=0 tx=0).stop 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 shutdown_connections 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55bdb38c3b90 0x55bdb38c5fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bdb38490e0 0x55bdb3842370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55bdb383d150 0x55bdb3851560 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 --2- 172.21.15.107:0/2556174725 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55bdb383c830 0x55bdb3851020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 >> 172.21.15.107:0/2556174725 conn(0x55bdb383a660 msgr2=0x55bdb3845fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.686+0000 7f8be4ff9640 0 client.25841 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 shutdown_connections 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:42.695 INFO:teuthology.orchestra.run.smithi107.stdout: "status": "no active scrubs running", 2024-08-20T06:43:42.696 INFO:teuthology.orchestra.run.smithi107.stdout: "scrubs": {} 2024-08-20T06:43:42.696 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c05c6f740 1 -- 172.21.15.107:0/2556174725 wait complete. 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c05c6f740 20 client.25841 trim_cache size 0 max 16384 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 msgr2=0x7f8be80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 0x7f8be80b6090 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005fd0 tx=0x7f8bf4009500 comp rx=0 tx=0).stop 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 msgr2=0x7f8bfc199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00e9f0 tx=0x7f8bec00edf0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 shutdown_connections 2024-08-20T06:43:42.697 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f8be80b3bd0 0x7f8be80b6090 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bfc107d30 0x7f8bfc19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bfc1073b0 0x7f8bfc199400 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 --2- 172.21.15.107:0/1096113550 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8bfc1019a0 0x7f8bfc198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.687+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 >> 172.21.15.107:0/1096113550 conn(0x7f8bfc0fd410 msgr2=0x7f8bfc06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.688+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 shutdown_connections 2024-08-20T06:43:42.698 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:42.688+0000 7f8c03876640 1 -- 172.21.15.107:0/1096113550 wait complete. 2024-08-20T06:43:43.305 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:43 smithi107 ceph-mon[18495]: pgmap v533: 97 pgs: 97 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-08-20T06:43:43.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:43 smithi183 ceph-mon[22204]: pgmap v533: 97 pgs: 97 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-08-20T06:43:43.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-08-20T06:43:43.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:43:43.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:43 smithi193 ceph-mon[21944]: pgmap v533: 97 pgs: 97 active+clean; 235 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 411 op/s 2024-08-20T06:43:43.780 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:44.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 -- 172.21.15.107:0/1939011117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341042c0 msgr2=0x7f453410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:44.301 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 --2- 172.21.15.107:0/1939011117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341042c0 0x7f453410c7b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4528009900 tx=0x7f452802a410 comp rx=0 tx=0).stop 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 -- 172.21.15.107:0/1939011117 shutdown_connections 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 --2- 172.21.15.107:0/1939011117 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f453410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 --2- 172.21.15.107:0/1939011117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341042c0 0x7f453410c7b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 --2- 172.21.15.107:0/1939011117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341039a0 0x7f4534103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 -- 172.21.15.107:0/1939011117 >> 172.21.15.107:0/1939011117 conn(0x7f45340fd400 msgr2=0x7f45340ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 -- 172.21.15.107:0/1939011117 shutdown_connections 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.284+0000 7f453a524640 1 -- 172.21.15.107:0/1939011117 wait complete. 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 Processor -- start 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 -- start start 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:44.302 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341042c0 0x7f453410c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f45341a1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112920 con 0x7f45341039a0 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f45341127a0 con 0x7f453410ce00 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f453a524640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f4534112aa0 con 0x7f45341042c0 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.285+0000 7f4533fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40202/0 (socket says 172.21.15.107:40202) 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 -- 172.21.15.107:0/1049856927 learned_addr learned my addr 172.21.15.107:0/1049856927 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4538ac1640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f45341a1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f45337fe640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341042c0 0x7f453410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 -- 172.21.15.107:0/1049856927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341042c0 msgr2=0x7f453410c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341042c0 0x7f453410c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 -- 172.21.15.107:0/1049856927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 msgr2=0x7f45341a1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f45341a1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 -- 172.21.15.107:0/1049856927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45341a22d0 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4538ac1640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f45341a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f4533fff640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f452400c920 tx=0x7f452400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524007c70 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45341a25c0 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f4524007e10 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240186f0 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.286+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45341abf30 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.287+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4534103d80 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.289+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f452400f450 con 0x7f45341039a0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.289+0000 7f45317fa640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 0x7f451c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.290+0000 7f45337fe640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 0x7f451c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:44.304 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.290+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f45240e3fa0 con 0x7f45341039a0 2024-08-20T06:43:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.290+0000 7f45337fe640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 0x7f451c0b6090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4528006010 tx=0x7f4528032ce0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:44.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.293+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4524014030 con 0x7f45341039a0 2024-08-20T06:43:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.535+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45341a0600 con 0x7f45341039a0 2024-08-20T06:43:44.536 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.536+0000 7f45317fa640 1 -- 172.21.15.107:0/1049856927 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11940 (secure 0 0 0) 0x7f45240ab720 con 0x7f45341039a0 2024-08-20T06:43:44.537 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:43:44.537 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":31,"btime":"2024-08-20T06:43:39:171082+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:43:38.678203+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24572,24562,24548]},"id":1}]} 2024-08-20T06:43:44.537 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 31 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 msgr2=0x7f451c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 0x7f451c0b6090 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4528006010 tx=0x7f4528032ce0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 msgr2=0x7f453410bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f452400c920 tx=0x7f452400cde0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 shutdown_connections 2024-08-20T06:43:44.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f453410ce00 0x7f45341a1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f451c0b3bd0 0x7f451c0b6090 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45341042c0 0x7f453410c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 --2- 172.21.15.107:0/1049856927 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f45341039a0 0x7f453410bbd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 >> 172.21.15.107:0/1049856927 conn(0x7f45340fd400 msgr2=0x7f45340ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 shutdown_connections 2024-08-20T06:43:44.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:44.538+0000 7f453a524640 1 -- 172.21.15.107:0/1049856927 wait complete. 2024-08-20T06:43:45.245 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-08-20T06:43:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:45 smithi183 ceph-mon[22204]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-08-20T06:43:45.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:45 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1049856927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:43:45.499 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:45 smithi193 ceph-mon[21944]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-08-20T06:43:45.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:45 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1049856927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:43:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:45 smithi107 ceph-mon[18495]: pgmap v534: 97 pgs: 97 active+clean; 215 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 464 op/s 2024-08-20T06:43:45.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:45 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1049856927' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:43:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- 172.21.15.107:0/3150409968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c1039a0 msgr2=0x7f1a8c103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.004 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 --2- 172.21.15.107:0/3150409968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c103d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c009900 tx=0x7f1a7c02a410 comp rx=0 tx=0).stop 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- 172.21.15.107:0/3150409968 shutdown_connections 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 --2- 172.21.15.107:0/3150409968 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 --2- 172.21.15.107:0/3150409968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 0x7f1a8c10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 --2- 172.21.15.107:0/3150409968 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c103d80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- 172.21.15.107:0/3150409968 >> 172.21.15.107:0/3150409968 conn(0x7f1a8c0fd400 msgr2=0x7f1a8c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- 172.21.15.107:0/3150409968 shutdown_connections 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- 172.21.15.107:0/3150409968 wait complete. 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 Processor -- start 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.971+0000 7f1a92968640 1 -- start start 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 0x7f1a8c101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.005 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c112920 con 0x7f1a8c10ce00 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1127a0 con 0x7f1a8c1039a0 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c112aa0 con 0x7f1a8c1042c0 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47114/0 (socket says 172.21.15.107:47114) 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c101650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:42936/0 (socket says 172.21.15.107:42936) 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 -- 172.21.15.107:0/1300141966 learned_addr learned my addr 172.21.15.107:0/1300141966 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 0x7f1a8c101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 -- 172.21.15.107:0/1300141966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 msgr2=0x7f1a8c101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 0x7f1a8c101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 -- 172.21.15.107:0/1300141966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 msgr2=0x7f1a8c101650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c101650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.006 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 -- 172.21.15.107:0/1300141966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8c1a72f0 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a90f05640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009fd0 tx=0x7f1a8400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a897fa640 1 -- 172.21.15.107:0/1300141966 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84017070 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a897fa640 1 -- 172.21.15.107:0/1300141966 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a84003e00 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a897fa640 1 -- 172.21.15.107:0/1300141966 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84010460 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.972+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8c1a75e0 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.973+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8c1a7ad0 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a897fa640 1 -- 172.21.15.107:0/1300141966 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f1a84003970 con 0x7f1a8c10ce00 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 Processor -- start 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 -- start start 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a2243aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2243fe0 0x55e2a22463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.007 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2a21ae930 con 0x55e2a2237730 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2a219de30 con 0x55e2a2243fe0 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a94d61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2a21a1e70 con 0x55e2a2238050 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.975+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47130/0 (socket says 172.21.15.107:47130) 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 -- 172.21.15.107:0/1004688218 learned_addr learned my addr 172.21.15.107:0/1004688218 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2243fe0 0x55e2a22463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a2243aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 -- 172.21.15.107:0/1004688218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 msgr2=0x55e2a2243aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a2243aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 -- 172.21.15.107:0/1004688218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2243fe0 msgr2=0x55e2a22463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2243fe0 0x55e2a22463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 -- 172.21.15.107:0/1004688218 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a1efd650 con 0x55e2a2237730 2024-08-20T06:43:46.008 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a897fa640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 0x7f1a740b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1a80012840 tx=0x7f1a8000c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b793350566682f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 0x7f1a740b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a897fa640 1 -- 172.21.15.107:0/1300141966 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f1a840e3d90 con 0x7f1a8c10ce00 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a727fc640 1 -- 172.21.15.107:0/1004688218 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000aac0 con 0x55e2a2237730 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a727fc640 1 -- 172.21.15.107:0/1004688218 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a800054b0 con 0x55e2a2237730 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.976+0000 7f1a727fc640 1 -- 172.21.15.107:0/1004688218 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004c90 con 0x55e2a2237730 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- 172.21.15.107:0/1004688218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 msgr2=0x55e2a2237b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 0x7f1a740b6090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0057c0 tx=0x7f1a7c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1a80012840 tx=0x7f1a8000c400 comp rx=0 tx=0).stop 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- 172.21.15.107:0/1004688218 shutdown_connections 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2243fe0 0x55e2a22463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a2243aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 --2- 172.21.15.107:0/1004688218 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2237b10 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.009 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- 172.21.15.107:0/1004688218 >> 172.21.15.107:0/1004688218 conn(0x55e2a2235560 msgr2=0x55e2a2236700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- 172.21.15.107:0/1004688218 shutdown_connections 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- 172.21.15.107:0/1004688218 wait complete. 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.977+0000 7f1a94d61740 1 -- start start 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 Processor -- start 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2237730 0x55e2a2241010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a2241a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55e2a2000700 con 0x55e2a2243fe0 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55e2a2124810 con 0x55e2a2238050 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55e2a2246c90 con 0x55e2a2237730 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a94d61740 10 client.0 _ll_register_callbacks cb 0x55e2a2197700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.010 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a90f05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a2241a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:42962/0 (socket says 172.21.15.107:42962) 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a8b7fe640 1 -- 172.21.15.107:0/2863377289 learned_addr learned my addr 172.21.15.107:0/2863377289 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.978+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2237730 0x55e2a2241010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 -- 172.21.15.107:0/2863377289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2237730 msgr2=0x55e2a2241010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2237730 0x55e2a2241010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 -- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 msgr2=0x55e2a2241550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 -- 172.21.15.107:0/2863377289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a22500f0 con 0x55e2a2243fe0 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a90f05640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a2241a90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1a840e1890 tx=0x7f1a84003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2030d1f849cb3fc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:46.011 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84020070 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f1a840a6f70 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e2a214d150 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a840af2e0 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 10 client.16344 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2a2252ae0 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55e2a214d150 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1a840a6650 con 0x55e2a2243fe0 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 10 client.16344 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 10 client.16344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:43:46.012 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 20 client.16344 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a2253f40 0x55e2a2256330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 4 client.16344 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.979+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2a21b0880 con 0x55e2a2253f40 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.980+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a2253f40 0x55e2a2256330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.980+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a2253f40 0x55e2a2256330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=630aaf3b18c4ce3 server_cookie=12b37965bf009e52 in_seq=0 out_seq=0 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.980+0000 7f1a6b7fe640 10 client.16344 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2863377289 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2a21b0880 con 0x55e2a2253f40 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a6b7fe640 10 client.16344 handle_command_reply: tid=0 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 client.16344 shutdown 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 msgr2=0x55e2a2241a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a2241a90 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f1a840e1890 tx=0x7f1a84003d20 comp rx=0 tx=0).stop 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 shutdown_connections 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a2241a90 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.013 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a2253f40 0x55e2a2256330 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2241550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2863377289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2237730 0x55e2a2241010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a6b7fe640 0 client.16344 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 >> 172.21.15.107:0/2863377289 conn(0x55e2a2235560 msgr2=0x55e2a223b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 shutdown_connections 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 1 -- 172.21.15.107:0/2863377289 wait complete. 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.981+0000 7f1a94d61740 20 client.16344 trim_cache size 0 max 16384 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 Processor -- start 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 -- start start 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2237730 0x55e2a225bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a225c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.014 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1b2b30 con 0x55e2a2243fe0 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1b03a0 con 0x55e2a2237730 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a94d61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ae1d0 con 0x55e2a2238050 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47156/0 (socket says 172.21.15.107:47156) 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 -- 172.21.15.107:0/3097339861 learned_addr learned my addr 172.21.15.107:0/3097339861 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2237730 0x55e2a225bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a225c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 -- 172.21.15.107:0/3097339861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 msgr2=0x55e2a225c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a225c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 -- 172.21.15.107:0/3097339861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2237730 msgr2=0x55e2a225bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2237730 0x55e2a225bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 -- 172.21.15.107:0/3097339861 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2a2235fe0 con 0x55e2a2243fe0 2024-08-20T06:43:46.015 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a90f05640 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400dfd0 tx=0x7f1a840097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6f678cb034e9ce server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.995+0000 7f1a71ffb640 1 -- 172.21.15.107:0/3097339861 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a84020070 con 0x55e2a2243fe0 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a71ffb640 1 -- 172.21.15.107:0/3097339861 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a840b3d90 con 0x55e2a2243fe0 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a71ffb640 1 -- 172.21.15.107:0/3097339861 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a840a6650 con 0x55e2a2243fe0 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 -- 172.21.15.107:0/3097339861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 msgr2=0x55e2a224bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f1a8400dfd0 tx=0x7f1a840097f0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 -- 172.21.15.107:0/3097339861 shutdown_connections 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2243fe0 0x55e2a224bf10 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2238050 0x55e2a225c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 --2- 172.21.15.107:0/3097339861 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2237730 0x55e2a225bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 -- 172.21.15.107:0/3097339861 >> 172.21.15.107:0/3097339861 conn(0x55e2a2235560 msgr2=0x55e2a22344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 -- 172.21.15.107:0/3097339861 shutdown_connections 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.996+0000 7f1a94d61740 1 -- 172.21.15.107:0/3097339861 wait complete. 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 -- start start 2024-08-20T06:43:46.016 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 Processor -- start 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2243fe0 0x55e2a22bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1ae860 con 0x55e2a2237730 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1afcd0 con 0x55e2a2238050 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8c1aef30 con 0x55e2a2243fe0 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a94d61740 10 client.0 _ll_register_callbacks cb 0x7f1a84004330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47172/0 (socket says 172.21.15.107:47172) 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a8b7fe640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:42972/0 (socket says 172.21.15.107:42972) 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.997+0000 7f1a8bfff640 1 -- 172.21.15.107:0/2758109032 learned_addr learned my addr 172.21.15.107:0/2758109032 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:46.017 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 -- 172.21.15.107:0/2758109032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2243fe0 msgr2=0x55e2a22bcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2243fe0 0x55e2a22bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 -- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 msgr2=0x55e2a2249a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 -- 172.21.15.107:0/2758109032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a22bd110 con 0x55e2a2237730 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a8bfff640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006010 tx=0x7f1a7c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79be1a0c7849b23c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c054060 con 0x55e2a2237730 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a840b3d90 con 0x55e2a2237730 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f1a7c0035f0 con 0x55e2a2237730 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a7c035070 con 0x55e2a2237730 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a94d61740 10 client.16350 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:46.018 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e2a22be470 con 0x55e2a2237730 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.998+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f1a840b3d90 con 0x55e2a2237730 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1a7c059070 con 0x55e2a2237730 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 10 client.16350 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 10 client.16350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 20 client.16350 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a22bebd0 0x55e2a22c0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 4 client.16350 mds_command: new command op to 24548 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 --> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2a21ac320 con 0x55e2a22bebd0 2024-08-20T06:43:46.019 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a22bebd0 0x55e2a22c0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a8b7fe640 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a22bebd0 0x55e2a22c0ff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e352b509088faaea server_cookie=f619b6c04bf723b8 in_seq=0 out_seq=0 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:45.999+0000 7f1a6b7fe640 10 client.16350 ms_handle_connect on v2:172.21.15.183:6836/2933033317 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a6b7fe640 1 -- 172.21.15.107:0/2758109032 <== mds.2 v2:172.21.15.183:6836/2933033317 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2a21ac320 con 0x55e2a22bebd0 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a6b7fe640 10 client.16350 handle_command_reply: tid=0 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 client.16350 shutdown 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 msgr2=0x55e2a2249500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.020 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c006010 tx=0x7f1a7c009260 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 shutdown_connections 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e2a2243fe0 0x55e2a22bcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:6836/2933033317,v1:172.21.15.183:6837/2933033317] conn(0x55e2a22bebd0 0x55e2a22c0ff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55e2a2238050 0x55e2a2249a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 --2- 172.21.15.107:0/2758109032 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55e2a2237730 0x55e2a2249500 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 >> 172.21.15.107:0/2758109032 conn(0x55e2a2235560 msgr2=0x55e2a2242640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a6b7fe640 0 client.16350 ms_handle_reset on v2:172.21.15.183:6836/2933033317 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 shutdown_connections 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 1 -- 172.21.15.107:0/2758109032 wait complete. 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a94d61740 20 client.16350 trim_cache size 0 max 16384 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 msgr2=0x7f1a740b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 0x7f1a740b6090 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c0057c0 tx=0x7f1a7c0058d0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.021 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 msgr2=0x7f1a8c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.000+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009fd0 tx=0x7f1a8400b260 comp rx=0 tx=0).stop 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 shutdown_connections 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1a8c10ce00 0x7f1a8c1020d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f1a740b3bd0 0x7f1a740b6090 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a8c1042c0 0x7f1a8c101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 --2- 172.21.15.107:0/1300141966 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f1a8c1039a0 0x7f1a8c101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 >> 172.21.15.107:0/1300141966 conn(0x7f1a8c0fd400 msgr2=0x7f1a8c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 shutdown_connections 2024-08-20T06:43:46.022 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:46.001+0000 7f1a92968640 1 -- 172.21.15.107:0/1300141966 wait complete. 2024-08-20T06:43:46.829 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-08-20T06:43:47.086 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:47.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:47 smithi183 ceph-mon[22204]: pgmap v535: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 399 op/s 2024-08-20T06:43:47.499 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:47 smithi107 ceph-mon[18495]: pgmap v535: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 399 op/s 2024-08-20T06:43:47.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:47 smithi193 ceph-mon[21944]: pgmap v535: 97 pgs: 97 active+clean; 208 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 399 op/s 2024-08-20T06:43:47.608 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 -- 172.21.15.107:0/3269414415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 msgr2=0x7f63ac103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/3269414415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac103d80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f639c009900 tx=0x7f639c02a410 comp rx=0 tx=0).stop 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 -- 172.21.15.107:0/3269414415 shutdown_connections 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/3269414415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac10ce00 0x7f63ac10f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/3269414415 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac1042c0 0x7f63ac10c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/3269414415 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac103d80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 -- 172.21.15.107:0/3269414415 >> 172.21.15.107:0/3269414415 conn(0x7f63ac0fd400 msgr2=0x7f63ac0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 -- 172.21.15.107:0/3269414415 shutdown_connections 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.586+0000 7f63b3bd9640 1 -- 172.21.15.107:0/3269414415 wait complete. 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 Processor -- start 2024-08-20T06:43:47.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 -- start start 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 0x7f63ac199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac10ce00 0x7f63ac19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1129b0 con 0x7f63ac1039a0 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac112830 con 0x7f63ac10ce00 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b3bd9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac112b30 con 0x7f63ac1042c0 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b1975640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b1975640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47176/0 (socket says 172.21.15.107:47176) 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b1975640 1 -- 172.21.15.107:0/217070315 learned_addr learned my addr 172.21.15.107:0/217070315 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:47.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b2176640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac10ce00 0x7f63ac19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.587+0000 7f63b1174640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 0x7f63ac199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 -- 172.21.15.107:0/217070315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 msgr2=0x7f63ac199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 0x7f63ac199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 -- 172.21.15.107:0/217070315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac10ce00 msgr2=0x7f63ac19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac10ce00 0x7f63ac19e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 -- 172.21.15.107:0/217070315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac19efd0 con 0x7f63ac1039a0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1174640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 0x7f63ac199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b1975640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639c02a930 tx=0x7f639c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63aaffd640 1 -- 172.21.15.107:0/217070315 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c03f070 con 0x7f63ac1039a0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63aaffd640 1 -- 172.21.15.107:0/217070315 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f639c003e40 con 0x7f63ac1039a0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63aaffd640 1 -- 172.21.15.107:0/217070315 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c035030 con 0x7f63ac1039a0 2024-08-20T06:43:47.611 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac19f260 con 0x7f63ac1039a0 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.588+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac1a5980 con 0x7f63ac1039a0 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.590+0000 7f63aaffd640 1 -- 172.21.15.107:0/217070315 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f639c042070 con 0x7f63ac1039a0 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 Processor -- start 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 -- start start 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c20000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 0x561687c22930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x561687b86930 con 0x561687c0f880 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561687b75e30 con 0x561687c101a0 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561687b79e70 con 0x561687c20540 2024-08-20T06:43:47.612 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47180/0 (socket says 172.21.15.107:47180) 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b1975640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 -- 172.21.15.107:0/3300882726 learned_addr learned my addr 172.21.15.107:0/3300882726 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b1174640 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 0x561687c22930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63aaffd640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 0x7f63940b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 -- 172.21.15.107:0/3300882726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 msgr2=0x561687c22930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 0x561687c22930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 -- 172.21.15.107:0/3300882726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 msgr2=0x561687c20000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c20000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.591+0000 7f63b2176640 1 -- 172.21.15.107:0/3300882726 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616878d5650 con 0x561687c0f880 2024-08-20T06:43:47.613 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f63b1174640 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 0x561687c22930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f63b2176640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 0x7f63940b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f63aaffd640 1 -- 172.21.15.107:0/217070315 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f639c104380 con 0x7f63ac1039a0 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f63b2176640 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b90 tx=0x7f63a400e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab0e5fcfcd5bed2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f6393fff640 1 -- 172.21.15.107:0/3300882726 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4023070 con 0x561687c0f880 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f63b2176640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 0x7f63940b6110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f63a4020000 tx=0x7f63a401d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f6393fff640 1 -- 172.21.15.107:0/3300882726 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a401b820 con 0x561687c0f880 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.592+0000 7f6393fff640 1 -- 172.21.15.107:0/3300882726 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4013590 con 0x561687c0f880 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3300882726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 msgr2=0x561687c0fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b90 tx=0x7f63a400e5a0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3300882726 shutdown_connections 2024-08-20T06:43:47.614 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c20540 0x561687c22930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c20000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3300882726 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c0f880 0x561687c0fc60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3300882726 >> 172.21.15.107:0/3300882726 conn(0x561687c0d6b0 msgr2=0x561687c0e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3300882726 shutdown_connections 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3300882726 wait complete. 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 -- start start 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.593+0000 7f63b5fd2740 1 Processor -- start 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 0x561687c1dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x5616879d8700 con 0x561687c101a0 2024-08-20T06:43:47.615 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x561687afc810 con 0x561687c20540 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x561687c231f0 con 0x561687c0f880 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 10 client.0 _ll_register_callbacks cb 0x561687b6f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47190/0 (socket says 172.21.15.107:47190) 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 -- 172.21.15.107:0/2997008948 learned_addr learned my addr 172.21.15.107:0/2997008948 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1975640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b2176640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 0x561687c1dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 -- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 msgr2=0x561687c1d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 -- 172.21.15.107:0/2997008948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 msgr2=0x561687c1dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.616 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 0x561687c1dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 -- 172.21.15.107:0/2997008948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561687c2c620 con 0x561687c101a0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b2176640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 0x561687c1dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b1174640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63a0012720 tx=0x7f63a0012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76228659d0e56b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004d20 con 0x561687c101a0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63a0010b60 con 0x561687c101a0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.594+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561687b25150 con 0x561687c101a0 2024-08-20T06:43:47.617 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63a0010d60 con 0x561687c101a0 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 10 client.16362 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561687c2f030 con 0x561687c101a0 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x561687b25150 con 0x561687c101a0 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f63a0019030 con 0x561687c101a0 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 10 client.16362 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 10 client.16362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 20 client.16362 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c305b0 0x561687c329f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 4 client.16362 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561687b88880 con 0x561687c305b0 2024-08-20T06:43:47.618 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.595+0000 7f63b1975640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c305b0 0x561687c329f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.596+0000 7f63917fa640 10 client.16362 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.596+0000 7f63b1975640 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c305b0 0x561687c329f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34bc36190e17279e server_cookie=6cd466f2a6159179 in_seq=0 out_seq=0 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.596+0000 7f63917fa640 1 -- 172.21.15.107:0/2997008948 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561687b88880 con 0x561687c305b0 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.596+0000 7f63917fa640 10 client.16362 handle_command_reply: tid=0 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.596+0000 7f63b5fd2740 1 client.16362 shutdown 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 msgr2=0x561687c1da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f63a0012720 tx=0x7f63a0012be0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 shutdown_connections 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c305b0 0x561687c329f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c20540 0x561687c1dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.619 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c101a0 0x561687c1da60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/2997008948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63917fa640 0 client.16362 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 >> 172.21.15.107:0/2997008948 conn(0x561687c0d6b0 msgr2=0x561687c131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 shutdown_connections 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 1 -- 172.21.15.107:0/2997008948 wait complete. 2024-08-20T06:43:47.620 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.597+0000 7f63b5fd2740 20 client.16362 trim_cache size 0 max 16384 2024-08-20T06:43:47.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 Processor -- start 2024-08-20T06:43:47.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 -- start start 2024-08-20T06:43:47.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.625 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c0cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1ae7a0 con 0x561687c20540 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1ac010 con 0x561687c101a0 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1a9e40 con 0x561687c0f880 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47196/0 (socket says 172.21.15.107:47196) 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 -- 172.21.15.107:0/3094218466 learned_addr learned my addr 172.21.15.107:0/3094218466 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b1975640 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b1174640 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c0cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 -- 172.21.15.107:0/3094218466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 msgr2=0x561687c1afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.626 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 -- 172.21.15.107:0/3094218466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 msgr2=0x561687c0cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c0cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.610+0000 7f63b2176640 1 -- 172.21.15.107:0/3094218466 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561687c33070 con 0x561687c20540 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63b2176640 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b60 tx=0x7f63a4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f73c75476b17de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63937fe640 1 -- 172.21.15.107:0/3094218466 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4023040 con 0x561687c20540 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63937fe640 1 -- 172.21.15.107:0/3094218466 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a401db50 con 0x561687c20540 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63937fe640 1 -- 172.21.15.107:0/3094218466 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4027a40 con 0x561687c20540 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3094218466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 msgr2=0x561687c1f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.611+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f63a4009b60 tx=0x7f63a4004660 comp rx=0 tx=0).stop 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3094218466 shutdown_connections 2024-08-20T06:43:47.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c1f260 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c0cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/3094218466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c1afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3094218466 >> 172.21.15.107:0/3094218466 conn(0x561687c0d6b0 msgr2=0x561687c1eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3094218466 shutdown_connections 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 -- 172.21.15.107:0/3094218466 wait complete. 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 -- start start 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 Processor -- start 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c287d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.612+0000 7f63b5fd2740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1aa4d0 con 0x561687c20540 2024-08-20T06:43:47.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1ab940 con 0x561687c101a0 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b5fd2740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac1aaba0 con 0x561687c0f880 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b5fd2740 10 client.0 _ll_register_callbacks cb 0x7f63ac06b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b1174640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47212/0 (socket says 172.21.15.107:47212) 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 -- 172.21.15.107:0/832575647 learned_addr learned my addr 172.21.15.107:0/832575647 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b1975640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c287d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 -- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 msgr2=0x561687c287d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c287d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 -- 172.21.15.107:0/832575647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 msgr2=0x561687c28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 -- 172.21.15.107:0/832575647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561687c38250 con 0x561687c20540 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b2176640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e5e0 tx=0x7f63a40096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be190446406e2010 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63917fa640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4033070 con 0x561687c20540 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f63a402f810 con 0x561687c20540 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.613+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63a401db50 con 0x561687c20540 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f63a4023050 con 0x561687c20540 2024-08-20T06:43:47.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 10 client.16368 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561687c9af10 con 0x561687c20540 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f63a401db50 con 0x561687c20540 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f63a401b070 con 0x561687c20540 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 10 client.16368 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 10 client.16368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 20 client.16368 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c9b570 0x561687c9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 4 client.16368 mds_command: new command op to 24562 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 --> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561687b84320 con 0x561687c9b570 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.614+0000 7f63b1975640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c9b570 0x561687c9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:47.631 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b1975640 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c9b570 0x561687c9d990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9ff9f63f276fa79 server_cookie=9659861b45b4fd in_seq=0 out_seq=0 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63917fa640 10 client.16368 ms_handle_connect on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63917fa640 1 -- 172.21.15.107:0/832575647 <== mds.1 v2:172.21.15.193:6838/3581594961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561687b84320 con 0x561687c9b570 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63917fa640 10 client.16368 handle_command_reply: tid=0 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 client.16368 shutdown 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 msgr2=0x561687c37a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f63a400e5e0 tx=0x7f63a40096b0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 shutdown_connections 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:6838/3581594961,v1:172.21.15.193:6839/3581594961] conn(0x561687c9b570 0x561687c9d990 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x561687c20540 0x561687c37a50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x561687c101a0 0x561687c28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 --2- 172.21.15.107:0/832575647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561687c0f880 0x561687c287d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.632 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63917fa640 0 client.16368 ms_handle_reset on v2:172.21.15.193:6838/3581594961 2024-08-20T06:43:47.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 >> 172.21.15.107:0/832575647 conn(0x561687c0d6b0 msgr2=0x561687c131e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 shutdown_connections 2024-08-20T06:43:47.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 1 -- 172.21.15.107:0/832575647 wait complete. 2024-08-20T06:43:47.633 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.615+0000 7f63b5fd2740 20 client.16368 trim_cache size 0 max 16384 2024-08-20T06:43:47.633 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 msgr2=0x7f63940b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 0x7f63940b6110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f63a4020000 tx=0x7f63a401d000 comp rx=0 tx=0).stop 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 msgr2=0x7f63ac198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f639c02a930 tx=0x7f639c003c20 comp rx=0 tx=0).stop 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 shutdown_connections 2024-08-20T06:43:47.634 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f63ac10ce00 0x7f63ac19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f63940b3c50 0x7f63940b6110 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac1042c0 0x7f63ac199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 --2- 172.21.15.107:0/217070315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63ac1039a0 0x7f63ac198f20 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 >> 172.21.15.107:0/217070315 conn(0x7f63ac0fd400 msgr2=0x7f63ac10f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 shutdown_connections 2024-08-20T06:43:47.635 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:47.616+0000 7f63b3bd9640 1 -- 172.21.15.107:0/217070315 wait complete. 2024-08-20T06:43:47.699 INFO:tasks.workunit:Stopping ['suites/blogbench.sh'] on client.0... 2024-08-20T06:43:47.699 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-20T06:43:47.999 DEBUG:teuthology.parallel:result is None 2024-08-20T06:43:48.000 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-08-20T06:43:48.033 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-08-20T06:43:48.033 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-08-20T06:43:48.044 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-08-20T06:43:48.463 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-08-20T06:43:48.710 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 -- 172.21.15.107:0/3050360089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 msgr2=0x7f800c103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 --2- 172.21.15.107:0/3050360089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c103dc0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009900 tx=0x7f7ffc02a410 comp rx=0 tx=0).stop 2024-08-20T06:43:49.308 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 -- 172.21.15.107:0/3050360089 shutdown_connections 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 --2- 172.21.15.107:0/3050360089 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c10ce40 0x7f800c10f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 --2- 172.21.15.107:0/3050360089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c104300 0x7f800c10c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 --2- 172.21.15.107:0/3050360089 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c103dc0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 -- 172.21.15.107:0/3050360089 >> 172.21.15.107:0/3050360089 conn(0x7f800c0fd440 msgr2=0x7f800c0ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 -- 172.21.15.107:0/3050360089 shutdown_connections 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.283+0000 7f801105e640 1 -- 172.21.15.107:0/3050360089 wait complete. 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 Processor -- start 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 -- start start 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c104300 0x7f800c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 0x7f800c19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.309 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1129c0 con 0x7f800c1039e0 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f800c112840 con 0x7f800c104300 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f801105e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f800c112b40 con 0x7f800c10ce40 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47236/0 (socket says 172.21.15.107:47236) 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 -- 172.21.15.107:0/1302568281 learned_addr learned my addr 172.21.15.107:0/1302568281 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800b59e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 0x7f800c19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f8003fff640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c104300 0x7f800c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 -- 172.21.15.107:0/1302568281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 msgr2=0x7f800c19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 0x7f800c19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 -- 172.21.15.107:0/1302568281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c104300 msgr2=0x7f800c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c104300 0x7f800c199470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800ad9d640 1 -- 172.21.15.107:0/1302568281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c19efe0 con 0x7f800c1039e0 2024-08-20T06:43:49.310 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.284+0000 7f800b59e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 0x7f800c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f800ad9d640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009230 tx=0x7f7ffc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f8008d99640 1 -- 172.21.15.107:0/1302568281 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc03f030 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f8008d99640 1 -- 172.21.15.107:0/1302568281 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7ffc003a80 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c19f270 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f8008d99640 1 -- 172.21.15.107:0/1302568281 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc035070 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.285+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c1a5990 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.287+0000 7f8008d99640 1 -- 172.21.15.107:0/1302568281 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f7ffc042030 con 0x7f800c1039e0 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.287+0000 7f8013457740 1 Processor -- start 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.287+0000 7f8013457740 1 -- start start 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f67bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 0x55ea96f6a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.311 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea96ed2930 con 0x55ea96f5b880 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea96ec1e30 con 0x55ea96f68130 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8013457740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ea96ec5e70 con 0x55ea96f5c1a0 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47240/0 (socket says 172.21.15.107:47240) 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 -- 172.21.15.107:0/30043746 learned_addr learned my addr 172.21.15.107:0/30043746 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800ad9d640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f67bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8003fff640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 0x55ea96f6a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 -- 172.21.15.107:0/30043746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 msgr2=0x55ea96f67bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f67bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 -- 172.21.15.107:0/30043746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 msgr2=0x55ea96f6a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 0x55ea96f6a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.312 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 -- 172.21.15.107:0/30043746 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea96c21650 con 0x55ea96f5b880 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8003fff640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 0x55ea96f6a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f8008d99640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 0x7f7ff80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800b59e640 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f800400c450 tx=0x7f800400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2192496d901a31e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.288+0000 7f800ad9d640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 0x7f7ff80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f8008d99640 1 -- 172.21.15.107:0/1302568281 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f7ffc104430 con 0x7f800c1039e0 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f80017fa640 1 -- 172.21.15.107:0/30043746 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004007430 con 0x55ea96f5b880 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f80017fa640 1 -- 172.21.15.107:0/30043746 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f8004007a60 con 0x55ea96f5b880 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f80017fa640 1 -- 172.21.15.107:0/30043746 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800400a920 con 0x55ea96f5b880 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f8013457740 1 -- 172.21.15.107:0/30043746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 msgr2=0x55ea96f5bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f800ad9d640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 0x7f7ff80b6090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0cb310 tx=0x7f7ffc107eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.289+0000 7f8013457740 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f800400c450 tx=0x7f800400c910 comp rx=0 tx=0).stop 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- 172.21.15.107:0/30043746 shutdown_connections 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f68130 0x55ea96f6a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.313 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f67bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- 172.21.15.107:0/30043746 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f5b880 0x55ea96f5bc60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- 172.21.15.107:0/30043746 >> 172.21.15.107:0/30043746 conn(0x55ea96f596b0 msgr2=0x55ea96f5a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- 172.21.15.107:0/30043746 shutdown_connections 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- 172.21.15.107:0/30043746 wait complete. 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- start start 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 Processor -- start 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f650c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x55ea96d24700 con 0x55ea96f68130 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x55ea96e48810 con 0x55ea96f5c1a0 2024-08-20T06:43:49.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x55ea96f6ade0 con 0x55ea96f5b880 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8013457740 10 client.0 _ll_register_callbacks cb 0x55ea96ebb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47242/0 (socket says 172.21.15.107:47242) 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f800b59e640 1 -- 172.21.15.107:0/1857713154 learned_addr learned my addr 172.21.15.107:0/1857713154 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.290+0000 7f8003fff640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 -- 172.21.15.107:0/1857713154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 msgr2=0x55ea96f650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.315 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 -- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 msgr2=0x55ea96f65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 -- 172.21.15.107:0/1857713154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea96f741c0 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f800b59e640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8004000f80 tx=0x7f8004004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e946c3eaa4da35c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f7ff27fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004028060 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f8004012040 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ea96e71150 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8004023460 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f8013457740 10 client.16380 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea96f76b70 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.291+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x55ea96e71150 con 0x55ea96f68130 2024-08-20T06:43:49.316 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8004021040 con 0x55ea96f68130 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 10 client.16380 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 20 client.16380 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96f78030 0x55ea96f7a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 4 client.16380 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea96ed4880 con 0x55ea96f78030 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8003fff640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96f78030 0x55ea96f7a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f8003fff640 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96f78030 0x55ea96f7a420 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882254d0eb55ce99 server_cookie=96692b01cb6ada68 in_seq=0 out_seq=0 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.292+0000 7f7ff27fc640 10 client.16380 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f7ff27fc640 1 -- 172.21.15.107:0/1857713154 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ea96ed4880 con 0x55ea96f78030 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f7ff27fc640 10 client.16380 handle_command_reply: tid=0 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 client.16380 shutdown 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 msgr2=0x55ea96f65b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.317 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8004000f80 tx=0x7f8004004b50 comp rx=0 tx=0).stop 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 shutdown_connections 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f65b40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96f78030 0x55ea96f7a420 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 --2- 172.21.15.107:0/1857713154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f7ff27fc640 0 client.16380 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 >> 172.21.15.107:0/1857713154 conn(0x55ea96f596b0 msgr2=0x55ea96f5f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 shutdown_connections 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 1 -- 172.21.15.107:0/1857713154 wait complete. 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.293+0000 7f8013457740 20 client.16380 trim_cache size 0 max 16384 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 Processor -- start 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 -- start start 2024-08-20T06:43:49.318 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5b880 0x55ea96f543d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f64950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1b0a20 con 0x55ea96f68130 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ae290 con 0x55ea96f5b880 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f8013457740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ac0c0 con 0x55ea96f5c1a0 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.306+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47252/0 (socket says 172.21.15.107:47252) 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 -- 172.21.15.107:0/3737672635 learned_addr learned my addr 172.21.15.107:0/3737672635 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5b880 0x55ea96f543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f8003fff640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f64950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 -- 172.21.15.107:0/3737672635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 msgr2=0x55ea96f64950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f64950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.319 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 -- 172.21.15.107:0/3737672635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5b880 msgr2=0x55ea96f543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5b880 0x55ea96f543d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 -- 172.21.15.107:0/3737672635 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea96f5a130 con 0x55ea96f68130 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f8003fff640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f64950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f800b59e640 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f800400a480 tx=0x7f80040070e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e8e1e3feeaabc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f8000ff9640 1 -- 172.21.15.107:0/3737672635 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004032020 con 0x55ea96f68130 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f8000ff9640 1 -- 172.21.15.107:0/3737672635 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f800400bf20 con 0x55ea96f68130 2024-08-20T06:43:49.320 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.307+0000 7f8000ff9640 1 -- 172.21.15.107:0/3737672635 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8004028070 con 0x55ea96f68130 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- 172.21.15.107:0/3737672635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 msgr2=0x55ea96f657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f800400a480 tx=0x7f80040070e0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- 172.21.15.107:0/3737672635 shutdown_connections 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f657a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5c1a0 0x55ea96f64950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- 172.21.15.107:0/3737672635 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5b880 0x55ea96f543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- 172.21.15.107:0/3737672635 >> 172.21.15.107:0/3737672635 conn(0x55ea96f596b0 msgr2=0x55ea96f65090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- 172.21.15.107:0/3737672635 shutdown_connections 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- 172.21.15.107:0/3737672635 wait complete. 2024-08-20T06:43:49.324 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- start start 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 Processor -- start 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f62cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.308+0000 7f8013457740 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ac750 con 0x55ea96f68130 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1adbc0 con 0x55ea96f5c1a0 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f800c1ace20 con 0x55ea96f5b880 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 10 client.0 _ll_register_callbacks cb 0x7f7ffc107d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T06:43:49.325 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8003fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47258/0 (socket says 172.21.15.107:47258) 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8003fff640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:43082/0 (socket says 172.21.15.107:43082) 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 -- 172.21.15.107:0/680913993 learned_addr learned my addr 172.21.15.107:0/680913993 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f62cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 -- 172.21.15.107:0/680913993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 msgr2=0x55ea96f62cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f62cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 -- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 msgr2=0x55ea96f700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 -- 172.21.15.107:0/680913993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea96f613b0 con 0x55ea96f68130 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8003fff640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f800b59e640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8004007980 tx=0x7f80040049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deae54b293bf89a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:49.326 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f7ff27fc640 10 client.0 ms_handle_connect on v2:172.21.15.107:3300/0 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800403d060 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mon.0 v2:172.21.15.107:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f800401b040 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800400bf20 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8004032030 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 10 client.16386 fetch_fsmap learned FSMap version 31 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.309+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ea96fe1d50 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mon.0 v2:172.21.15.107:3300/0 4 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f800400bf20 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mon.0 v2:172.21.15.107:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8004037040 con 0x55ea96f68130 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 10 client.16386 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 20 client.16386 populate_metadata read hostname 'smithi107' 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96fe2480 0x55ea96fe48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:49.327 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 4 client.16386 mds_command: new command op to 24572 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 --> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ea96ed0320 con 0x55ea96fe2480 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.310+0000 7f8003fff640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96fe2480 0x55ea96fe48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f7ff27fc640 10 client.16386 ms_handle_connect on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8003fff640 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96fe2480 0x55ea96fe48a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afd907d863c2cfa6 server_cookie=6aa25f62e9c0f13e in_seq=0 out_seq=0 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f7ff27fc640 1 -- 172.21.15.107:0/680913993 <== mds.0 v2:172.21.15.183:6838/2078682144 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ea96ed0320 con 0x55ea96fe2480 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f7ff27fc640 10 client.16386 handle_command_reply: tid=0 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 client.16386 shutdown 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 msgr2=0x55ea96f70830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8004007980 tx=0x7f80040049c0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 shutdown_connections 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x55ea96f68130 0x55ea96f70830 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x55ea96f5c1a0 0x55ea96f700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.328 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.183:6838/2078682144,v1:172.21.15.183:6839/2078682144] conn(0x55ea96fe2480 0x55ea96fe48a0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 --2- 172.21.15.107:0/680913993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55ea96f5b880 0x55ea96f62cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f7ff27fc640 0 client.16386 ms_handle_reset on v2:172.21.15.183:6838/2078682144 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 >> 172.21.15.107:0/680913993 conn(0x55ea96f596b0 msgr2=0x55ea96f658b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 shutdown_connections 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 1 -- 172.21.15.107:0/680913993 wait complete. 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.311+0000 7f8013457740 20 client.16386 trim_cache size 0 max 16384 2024-08-20T06:43:49.329 INFO:teuthology.orchestra.run.smithi107.stdout:[] 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 msgr2=0x7f7ff80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 0x7f7ff80b6090 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc0cb310 tx=0x7f7ffc107eb0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 msgr2=0x7f800c198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009230 tx=0x7f7ffc004270 comp rx=0 tx=0).stop 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 shutdown_connections 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7ff80b3bd0 0x7f7ff80b6090 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f800c10ce40 0x7f800c19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.330 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f800c104300 0x7f800c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 --2- 172.21.15.107:0/1302568281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f800c1039e0 0x7f800c198f30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:49.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 >> 172.21.15.107:0/1302568281 conn(0x7f800c0fd440 msgr2=0x7f800c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:49.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 shutdown_connections 2024-08-20T06:43:49.331 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:49.312+0000 7f801105e640 1 -- 172.21.15.107:0/1302568281 wait complete. 2024-08-20T06:43:49.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:49 smithi183 ceph-mon[22204]: pgmap v536: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 429 op/s 2024-08-20T06:43:49.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:49 smithi193 ceph-mon[21944]: pgmap v536: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 429 op/s 2024-08-20T06:43:49.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:49 smithi107 ceph-mon[18495]: pgmap v536: 97 pgs: 97 active+clean; 199 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 429 op/s 2024-08-20T06:43:50.157 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-08-20T06:43:50.158 INFO:tasks.fwd_scrub:done joining 2024-08-20T06:43:50.159 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-20T06:43:50.170 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-20T06:43:50.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/439503816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 msgr2=0x7fa2a4109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/439503816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4109a20 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009cd0 tx=0x7fa29c02a8e0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.272 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/439503816 shutdown_connections 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/439503816 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4109a20 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/439503816 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a41026e0 0x7fa2a4102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/439503816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a4101590 0x7fa2a4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/439503816 >> 172.21.15.107:0/439503816 conn(0x7fa2a40fce50 msgr2=0x7fa2a40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/439503816 shutdown_connections 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/439503816 wait complete. 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.269+0000 7fa2aa0fd640 1 Processor -- start 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 -- start start 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a4101590 0x7fa2a41977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 0x7fa2a4197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d100 con 0x7fa2a41030a0 2024-08-20T06:43:50.273 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410cf80 con 0x7fa2a4101590 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2aa0fd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a410d280 con 0x7fa2a41026e0 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47270/0 (socket says 172.21.15.107:47270) 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 -- 172.21.15.107:0/47216143 learned_addr learned my addr 172.21.15.107:0/47216143 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a2ffd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 0x7fa2a4197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a37fe640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a4101590 0x7fa2a41977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 -- 172.21.15.107:0/47216143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 msgr2=0x7fa2a4197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 0x7fa2a4197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 -- 172.21.15.107:0/47216143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a4101590 msgr2=0x7fa2a41977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a4101590 0x7fa2a41977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a3fff640 1 -- 172.21.15.107:0/47216143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a4199300 con 0x7fa2a41030a0 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.270+0000 7fa2a2ffd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 0x7fa2a4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:50.274 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2a3fff640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa29c003ba0 tx=0x7fa29c003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004240 con 0x7fa2a41030a0 2024-08-20T06:43:50.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa29c0305a0 con 0x7fa2a41030a0 2024-08-20T06:43:50.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0466e0 con 0x7fa2a41030a0 2024-08-20T06:43:50.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a4199590 con 0x7fa2a41030a0 2024-08-20T06:43:50.275 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.271+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a419fcc0 con 0x7fa2a41030a0 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.272+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a4108810 con 0x7fa2a41030a0 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.273+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fa29c030a50 con 0x7fa2a41030a0 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.274+0000 7fa2a0ff9640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 0x7fa2900b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.274+0000 7fa2a37fe640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 0x7fa2900b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.274+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa29c02e030 con 0x7fa2a41030a0 2024-08-20T06:43:50.276 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.274+0000 7fa2a37fe640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 0x7fa2900b6110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa294005fd0 tx=0x7fa294005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.277 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.277+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa29c04b060 con 0x7fa2a41030a0 2024-08-20T06:43:50.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.454+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2a402e590 con 0x7fa2a41030a0 2024-08-20T06:43:50.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.455+0000 7fa2a0ff9640 1 -- 172.21.15.107:0/47216143 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7fa29c0cc8d0 con 0x7fa2a41030a0 2024-08-20T06:43:50.457 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:43:50.457 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:43:50.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 msgr2=0x7fa2900b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 0x7fa2900b6110 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa294005fd0 tx=0x7fa294005850 comp rx=0 tx=0).stop 2024-08-20T06:43:50.459 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 msgr2=0x7fa2a4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa29c003ba0 tx=0x7fa29c003bd0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 shutdown_connections 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa2a41030a0 0x7fa2a4198a40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa2900b3c50 0x7fa2900b6110 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa2a41026e0 0x7fa2a4197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 --2- 172.21.15.107:0/47216143 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa2a4101590 0x7fa2a41977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 >> 172.21.15.107:0/47216143 conn(0x7fa2a40fce50 msgr2=0x7fa2a41059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 shutdown_connections 2024-08-20T06:43:50.460 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.458+0000 7fa2aa0fd640 1 -- 172.21.15.107:0/47216143 wait complete. 2024-08-20T06:43:50.470 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T06:30:39.511744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-20T06:34:56.266774+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-20T06:34:58.530080+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T06:43:50.471 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.571+0000 7f7a209aa640 1 -- 172.21.15.107:0/3620653596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c103080 msgr2=0x7f7a1c109a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.571+0000 7f7a209aa640 1 --2- 172.21.15.107:0/3620653596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c103080 0x7f7a1c109a00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7a14009cd0 tx=0x7f7a1402a8e0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- 172.21.15.107:0/3620653596 shutdown_connections 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- 172.21.15.107:0/3620653596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c103080 0x7f7a1c109a00 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- 172.21.15.107:0/3620653596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- 172.21.15.107:0/3620653596 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c101570 0x7f7a1c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.575 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- 172.21.15.107:0/3620653596 >> 172.21.15.107:0/3620653596 conn(0x7f7a1c0fce50 msgr2=0x7f7a1c0ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- 172.21.15.107:0/3620653596 shutdown_connections 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- 172.21.15.107:0/3620653596 wait complete. 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 Processor -- start 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- start start 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c101570 0x7f7a1c06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 0x7f7a1c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10d0e0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10cf60 con 0x7f7a1c103080 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.572+0000 7f7a209aa640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c10d260 con 0x7f7a1c101570 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47280/0 (socket says 172.21.15.107:47280) 2024-08-20T06:43:50.576 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 -- 172.21.15.107:0/920312147 learned_addr learned my addr 172.21.15.107:0/920312147 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a1a59c640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c101570 0x7f7a1c06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a1ad9d640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 0x7f7a1c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 -- 172.21.15.107:0/920312147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c101570 msgr2=0x7f7a1c06a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c101570 0x7f7a1c06a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 -- 172.21.15.107:0/920312147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 msgr2=0x7f7a1c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 0x7f7a1c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 -- 172.21.15.107:0/920312147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c1a0550 con 0x7f7a1c1026c0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a1ad9d640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 0x7f7a1c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a13fff640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400d770 tx=0x7f7a0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04017070 con 0x7f7a1c1026c0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7a040054b0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1a0840 con 0x7f7a1c1026c0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04004c60 con 0x7f7a1c1026c0 2024-08-20T06:43:50.577 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.573+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a0d40 con 0x7f7a1c1026c0 2024-08-20T06:43:50.578 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.574+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c1088e0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.575+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f7a04004e00 con 0x7f7a1c1026c0 2024-08-20T06:43:50.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.576+0000 7f7a137fe640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 0x7f7a080b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.576+0000 7f7a1a59c640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 0x7f7a080b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.576+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f7a040e36a0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.579 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.577+0000 7f7a1a59c640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 0x7f7a080b6090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00bbe0 tx=0x7f7a0c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.580+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7a040aacf0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.756 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.755+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a1c063a80 con 0x7f7a1c1026c0 2024-08-20T06:43:50.757 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.756+0000 7f7a137fe640 1 -- 172.21.15.107:0/920312147 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7f7a040aa5f0 con 0x7f7a1c1026c0 2024-08-20T06:43:50.757 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-08-20T06:43:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 msgr2=0x7f7a080b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 0x7f7a080b6090 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c00bbe0 tx=0x7f7a0c00b390 comp rx=0 tx=0).stop 2024-08-20T06:43:50.759 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 msgr2=0x7f7a1c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7a0400d770 tx=0x7f7a0400dc30 comp rx=0 tx=0).stop 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 shutdown_connections 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7a1c103080 0x7f7a1c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7a1c1026c0 0x7f7a1c19d6a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7a080b3bd0 0x7f7a080b6090 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 --2- 172.21.15.107:0/920312147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a1c101570 0x7f7a1c06a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 >> 172.21.15.107:0/920312147 conn(0x7f7a1c0fce50 msgr2=0x7f7a1c105ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 shutdown_connections 2024-08-20T06:43:50.760 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.758+0000 7f7a209aa640 1 -- 172.21.15.107:0/920312147 wait complete. 2024-08-20T06:43:50.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-20T06:43:50.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 -- 172.21.15.107:0/2118145724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 msgr2=0x7f3540107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 --2- 172.21.15.107:0/2118145724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540107c40 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3534009900 tx=0x7f353402a410 comp rx=0 tx=0).stop 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 -- 172.21.15.107:0/2118145724 shutdown_connections 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 --2- 172.21.15.107:0/2118145724 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f354010a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 --2- 172.21.15.107:0/2118145724 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540107c40 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 --2- 172.21.15.107:0/2118145724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 0x7f3540105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 -- 172.21.15.107:0/2118145724 >> 172.21.15.107:0/2118145724 conn(0x7f35400fce20 msgr2=0x7f35400ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 -- 172.21.15.107:0/2118145724 shutdown_connections 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.870+0000 7f354530f640 1 -- 172.21.15.107:0/2118145724 wait complete. 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 Processor -- start 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 -- start start 2024-08-20T06:43:50.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 0x7f35401989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f3540199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f354010e180 con 0x7f35401057d0 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f354010e000 con 0x7f3540108380 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f354530f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f354010e300 con 0x7f354006bb80 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f353e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f353e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47300/0 (socket says 172.21.15.107:47300) 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f353e7fc640 1 -- 172.21.15.107:0/1563027256 learned_addr learned my addr 172.21.15.107:0/1563027256 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.871+0000 7f353f7fe640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f3540199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 -- 172.21.15.107:0/1563027256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 msgr2=0x7f35401989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353effd640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 0x7f35401989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 0x7f35401989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 -- 172.21.15.107:0/1563027256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 msgr2=0x7f3540199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f3540199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 -- 172.21.15.107:0/1563027256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354019a530 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353f7fe640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f3540199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f353e7fc640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3534005ab0 tx=0x7f35340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353403f030 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f3534003c90 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354019a7c0 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3534035070 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.872+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354019aad0 con 0x7f35401057d0 2024-08-20T06:43:50.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.873+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35400fe910 con 0x7f35401057d0 2024-08-20T06:43:50.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.874+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f3534042030 con 0x7f35401057d0 2024-08-20T06:43:50.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.875+0000 7f3527fff640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 0x7f35280b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:50.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.875+0000 7f353effd640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 0x7f35280b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:50.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.875+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f35341046a0 con 0x7f35401057d0 2024-08-20T06:43:50.879 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.876+0000 7f353effd640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 0x7f35280b6090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3530005fd0 tx=0x7f3530005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:50.880 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:50.880+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f35340cbcb0 con 0x7f35401057d0 2024-08-20T06:43:51.055 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.054+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3540063a80 con 0x7f35401057d0 2024-08-20T06:43:51.056 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.055+0000 7f3527fff640 1 -- 172.21.15.107:0/1563027256 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v73) ==== 104+0+11 (secure 0 0 0) 0x7f35340cb5b0 con 0x7f35401057d0 2024-08-20T06:43:51.056 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 32 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 msgr2=0x7f35280b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 0x7f35280b6090 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3530005fd0 tx=0x7f3530005850 comp rx=0 tx=0).stop 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 msgr2=0x7f3540199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3534005ab0 tx=0x7f35340042f0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 shutdown_connections 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f3540108380 0x7f3540199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f35401057d0 0x7f3540199040 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f35280b3bd0 0x7f35280b6090 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 --2- 172.21.15.107:0/1563027256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f354006bb80 0x7f35401989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 >> 172.21.15.107:0/1563027256 conn(0x7f35400fce20 msgr2=0x7f3540103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:51.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 shutdown_connections 2024-08-20T06:43:51.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.057+0000 7f354530f640 1 -- 172.21.15.107:0/1563027256 wait complete. 2024-08-20T06:43:51.069 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-20T06:43:51.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 -- 172.21.15.107:0/2194863792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1030b0 msgr2=0x7f492c109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.173 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2194863792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1030b0 0x7f492c109a30 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f4924009cd0 tx=0x7f492402a8e0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 -- 172.21.15.107:0/2194863792 shutdown_connections 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2194863792 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1030b0 0x7f492c109a30 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2194863792 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 0x7f492c102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2194863792 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1015a0 0x7f492c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 -- 172.21.15.107:0/2194863792 >> 172.21.15.107:0/2194863792 conn(0x7f492c0fce40 msgr2=0x7f492c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 -- 172.21.15.107:0/2194863792 shutdown_connections 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.170+0000 7f4930e3f640 1 -- 172.21.15.107:0/2194863792 wait complete. 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 Processor -- start 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 -- start start 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.174 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 0x7f492c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1030b0 0x7f492c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d0c0 con 0x7f492c1015a0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10cf40 con 0x7f492c1030b0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f4930e3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d240 con 0x7f492c1026f0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f492a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f492a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47322/0 (socket says 172.21.15.107:47322) 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.171+0000 7f492a59c640 1 -- 172.21.15.107:0/2428243732 learned_addr learned my addr 172.21.15.107:0/2428243732 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492ad9d640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1030b0 0x7f492c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f4923fff640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 0x7f492c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 -- 172.21.15.107:0/2428243732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 msgr2=0x7f492c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 0x7f492c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 -- 172.21.15.107:0/2428243732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1030b0 msgr2=0x7f492c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1030b0 0x7f492c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.175 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 -- 172.21.15.107:0/2428243732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c199400 con 0x7f492c1015a0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f492a59c640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e9f0 tx=0x7f491c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c00cd80 con 0x7f492c1015a0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f491c0108c0 con 0x7f492c1015a0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c1996f0 con 0x7f492c1015a0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491c0187c0 con 0x7f492c1015a0 2024-08-20T06:43:51.176 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.172+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c19fdc0 con 0x7f492c1015a0 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.173+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c108990 con 0x7f492c1015a0 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.174+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f491c010430 con 0x7f492c1015a0 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.175+0000 7f49237fe640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.175+0000 7f4923fff640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.175+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f491c0e44d0 con 0x7f492c1015a0 2024-08-20T06:43:51.177 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.176+0000 7f4923fff640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4914009b60 tx=0x7f4914005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:51.179 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.178+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f491c0abb20 con 0x7f492c1015a0 2024-08-20T06:43:51.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.355+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f492c063a80 con 0x7f492c1015a0 2024-08-20T06:43:51.356 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.355+0000 7f49237fe640 1 -- 172.21.15.107:0/2428243732 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v73) ==== 100+0+11 (secure 0 0 0) 0x7f491c0ab420 con 0x7f492c1015a0 2024-08-20T06:43:51.357 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 64 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 msgr2=0x7f49180b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4914009b60 tx=0x7f4914005f90 comp rx=0 tx=0).stop 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 msgr2=0x7f492c1978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f491c00e9f0 tx=0x7f491c00eeb0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 shutdown_connections 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f492c1030b0 0x7f492c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f49180b3bd0 0x7f49180b6090 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f492c1026f0 0x7f492c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.359 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 --2- 172.21.15.107:0/2428243732 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f492c1015a0 0x7f492c1978c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 >> 172.21.15.107:0/2428243732 conn(0x7f492c0fce40 msgr2=0x7f492c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:51.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 shutdown_connections 2024-08-20T06:43:51.360 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.358+0000 7f4930e3f640 1 -- 172.21.15.107:0/2428243732 wait complete. 2024-08-20T06:43:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:51 smithi107 ceph-mon[18495]: pgmap v537: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-08-20T06:43:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/47216143' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:43:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/920312147' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:51 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1563027256' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.370 DEBUG:teuthology.orchestra.run.smithi107:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-20T06:43:51.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 -- 172.21.15.107:0/198387281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194103ba0 msgr2=0x7fa194103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 --2- 172.21.15.107:0/198387281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194103ba0 0x7fa194103fa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa184009900 tx=0x7fa18402a410 comp rx=0 tx=0).stop 2024-08-20T06:43:51.471 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 -- 172.21.15.107:0/198387281 shutdown_connections 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 --2- 172.21.15.107:0/198387281 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa194102db0 0x7fa19410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 --2- 172.21.15.107:0/198387281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1941023f0 0x7fa194102870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 --2- 172.21.15.107:0/198387281 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194103ba0 0x7fa194103fa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 -- 172.21.15.107:0/198387281 >> 172.21.15.107:0/198387281 conn(0x7fa1940fdf40 msgr2=0x7fa194100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 -- 172.21.15.107:0/198387281 shutdown_connections 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.468+0000 7fa198c85640 1 -- 172.21.15.107:0/198387281 wait complete. 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 Processor -- start 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 -- start start 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1941023f0 0x7fa194198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 0x7fa194199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410e330 con 0x7fa194102db0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410e1b0 con 0x7fa1941023f0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa198c85640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fa19410e4b0 con 0x7fa194103ba0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47350/0 (socket says 172.21.15.107:47350) 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 -- 172.21.15.107:0/1719977820 learned_addr learned my addr 172.21.15.107:0/1719977820 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa192d9d640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 0x7fa194199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa19259c640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1941023f0 0x7fa194198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 -- 172.21.15.107:0/1719977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 msgr2=0x7fa194199d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 0x7fa194199d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 -- 172.21.15.107:0/1719977820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1941023f0 msgr2=0x7fa194198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1941023f0 0x7fa194198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 -- 172.21.15.107:0/1719977820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19419a630 con 0x7fa194102db0 2024-08-20T06:43:51.473 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa192d9d640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 0x7fa194199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.469+0000 7fa191d9b640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c970 tx=0x7fa18800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.470+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188007bb0 con 0x7fa194102db0 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.470+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fa188007d50 con 0x7fa194102db0 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.470+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19419a920 con 0x7fa194102db0 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.470+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188004e40 con 0x7fa194102db0 2024-08-20T06:43:51.474 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.470+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941a0ff0 con 0x7fa194102db0 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.471+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa194102870 con 0x7fa194102db0 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.472+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fa188005000 con 0x7fa194102db0 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.473+0000 7fa1837fe640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 0x7fa17c0b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.473+0000 7fa19259c640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 0x7fa17c0b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.473+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fa1880e3dd0 con 0x7fa194102db0 2024-08-20T06:43:51.475 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.473+0000 7fa19259c640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 0x7fa17c0b6090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa184000c00 tx=0x7fa184031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:43:51.476 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.476+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa188014030 con 0x7fa194102db0 2024-08-20T06:43:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:51 smithi183 ceph-mon[22204]: pgmap v537: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-08-20T06:43:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/47216143' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:43:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/920312147' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:51 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1563027256' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:51 smithi193 ceph-mon[21944]: pgmap v537: 97 pgs: 97 active+clean; 184 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 416 op/s 2024-08-20T06:43:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/47216143' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:43:51.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/920312147' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:51 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1563027256' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T06:43:51.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.704+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fa19402e340 con 0x7fa194102db0 2024-08-20T06:43:51.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.711+0000 7fa1837fe640 1 -- 172.21.15.107:0/1719977820 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+821386 (secure 0 0 0) 0x7fa1880ab420 con 0x7fa194102db0 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "epoch": 20, 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "active_gid": 14150, 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "active_name": "x", 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addrs": { 2024-08-20T06:43:51.725 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6800", 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1083091367 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v1", 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:6801", 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1083091367 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.726 INFO:teuthology.orchestra.run.smithi107.stdout: "active_addr": "172.21.15.107:6801/1083091367", 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "active_change": "2024-08-20T06:28:01.208713+0000", 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "active_mgr_features": 4540701547738038271, 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "available": true, 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "standbys": [ 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 24109, 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "z", 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.727 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T06:43:51.728 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.729 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.731 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.732 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.735 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-08-20T06:43:51.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-08-20T06:43:51.738 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-08-20T06:43:51.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.741 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.742 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.743 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.744 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-08-20T06:43:51.746 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.747 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T06:43:51.748 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-08-20T06:43:51.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.754 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T06:43:51.755 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.756 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.759 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-08-20T06:43:51.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-08-20T06:43:51.763 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.764 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:51.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.767 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-08-20T06:43:51.769 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T06:43:51.770 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.771 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-08-20T06:43:51.772 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-08-20T06:43:51.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-08-20T06:43:51.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-08-20T06:43:51.777 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-08-20T06:43:51.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.780 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.781 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-08-20T06:43:51.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.784 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.787 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-08-20T06:43:51.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-08-20T06:43:51.789 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.790 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.792 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.793 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.794 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nginx": { 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nginx", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T06:43:51.795 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nginx container image", 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-08-20T06:43:51.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.797 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:51.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.800 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.801 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-08-20T06:43:51.803 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-08-20T06:43:51.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.805 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.806 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.809 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.811 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:51.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.814 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.816 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.820 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T06:43:51.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:51.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.831 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T06:43:51.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.834 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T06:43:51.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-08-20T06:43:51.837 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.838 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.839 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.840 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.843 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.844 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-08-20T06:43:51.846 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.847 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.848 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T06:43:51.849 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.851 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.852 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.853 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.854 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.855 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.856 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-08-20T06:43:51.857 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-08-20T06:43:51.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.862 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.863 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.864 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T06:43:51.869 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T06:43:51.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.871 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.872 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.873 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T06:43:51.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.876 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.879 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.882 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.884 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T06:43:51.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.887 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.888 INFO:teuthology.orchestra.run.smithi107.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "[]", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.891 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.892 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T06:43:51.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.896 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.898 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T06:43:51.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T06:43:51.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.907 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T06:43:51.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:51.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T06:43:51.912 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T06:43:51.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.915 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.919 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.920 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-08-20T06:43:51.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.928 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.929 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.930 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.933 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.935 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.937 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.938 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.939 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.940 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.943 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.944 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.945 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.947 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.948 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.951 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.954 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.961 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.962 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.964 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-08-20T06:43:51.965 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.966 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.969 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.972 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.975 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.978 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-08-20T06:43:51.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-08-20T06:43:51.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-08-20T06:43:51.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.983 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.984 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.989 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.990 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.993 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:51.995 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:51.996 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:51.997 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:51.998 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:51.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.000 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.003 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.004 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T06:43:52.005 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.006 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.009 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.010 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.011 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.012 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.013 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.014 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.015 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.016 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.017 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.018 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.020 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-08-20T06:43:52.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.023 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-08-20T06:43:52.024 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.027 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.030 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.031 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.033 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-08-20T06:43:52.036 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.037 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.038 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.040 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.042 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.043 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.044 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.045 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.047 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.048 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.051 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.053 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.054 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.058 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-08-20T06:43:52.061 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-08-20T06:43:52.063 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.064 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.065 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.066 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.067 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.069 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-08-20T06:43:52.070 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.071 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.073 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.074 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.076 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.077 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.078 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.080 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.081 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-08-20T06:43:52.086 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.087 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:52.089 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.090 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.091 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.095 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.096 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.097 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.098 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-08-20T06:43:52.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T06:43:52.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.107 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.109 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.114 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-08-20T06:43:52.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.117 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.118 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:52.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.120 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.121 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.122 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.123 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.124 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.125 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.126 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-08-20T06:43:52.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.128 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.129 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.130 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.136 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.137 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.138 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.139 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.140 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:52.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.142 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.146 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.148 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.149 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.152 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.154 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.156 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.157 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.158 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.159 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.162 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.164 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.165 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.167 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.169 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.170 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.171 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.172 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.173 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-08-20T06:43:52.174 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-08-20T06:43:52.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.176 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.177 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.180 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smb", 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "internal_store_backend": { 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "internal_store_backend", 2024-08-20T06:43:52.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.185 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.186 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.187 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.188 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.189 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.192 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "update_orchestration": { 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "update_orchestration", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.194 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.195 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.198 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.199 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.200 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.201 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.203 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.204 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.205 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.206 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.207 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.208 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.209 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.216 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.220 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.221 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.222 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.224 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.225 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.228 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.229 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.230 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.231 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.233 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.234 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.235 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.236 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.237 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.238 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-08-20T06:43:52.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-08-20T06:43:52.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.246 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.249 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.250 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.252 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.253 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.256 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.257 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.258 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.260 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.261 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.263 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.266 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.268 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.270 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.271 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.272 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-08-20T06:43:52.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.276 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T06:43:52.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-08-20T06:43:52.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.281 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.282 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.284 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.286 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.287 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-08-20T06:43:52.291 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.292 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.293 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "gid": 24131, 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "y", 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "mgr_features": 4540701547738038271, 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-08-20T06:43:52.294 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.295 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.296 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.298 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.299 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.300 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.301 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-08-20T06:43:52.303 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.304 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.305 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.306 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.307 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.310 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.312 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.314 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.315 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T06:43:52.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-08-20T06:43:52.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.322 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.323 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.326 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-08-20T06:43:52.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:52.329 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-08-20T06:43:52.330 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.331 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.333 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.337 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:52.338 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-08-20T06:43:52.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.345 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.346 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.347 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.350 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.351 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T06:43:52.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.354 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.356 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-08-20T06:43:52.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-08-20T06:43:52.360 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.361 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nginx": { 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nginx", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nginx container image", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.364 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.367 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.369 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.370 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.371 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-08-20T06:43:52.374 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.376 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T06:43:52.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.380 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.381 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.382 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.383 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.384 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.385 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.387 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.389 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T06:43:52.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.396 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-08-20T06:43:52.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.398 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T06:43:52.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.404 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.406 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-08-20T06:43:52.409 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-08-20T06:43:52.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.416 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.417 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.420 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.422 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T06:43:52.425 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T06:43:52.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T06:43:52.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.428 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.429 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.431 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.436 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.437 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.442 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.448 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.448 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.449 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "[]", 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-08-20T06:43:52.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T06:43:52.457 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.458 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.460 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.463 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.464 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T06:43:52.470 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T06:43:52.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T06:43:52.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.473 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.474 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.478 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T06:43:52.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-08-20T06:43:52.480 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.481 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.486 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.487 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.488 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.491 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:52.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:52.498 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.499 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:52.502 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.503 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.504 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.505 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.506 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.507 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.508 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.509 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.510 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T06:43:52.511 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.512 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-08-20T06:43:52.513 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.514 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.515 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.516 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.517 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.518 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.519 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-08-20T06:43:52.520 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.521 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.522 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.523 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.524 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.525 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.526 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.527 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:52.528 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-08-20T06:43:52.529 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.530 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.531 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.532 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.533 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.534 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.535 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-08-20T06:43:52.536 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.537 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.538 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.539 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.540 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.541 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.542 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.543 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.544 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:52.545 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:52.546 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.547 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.548 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-08-20T06:43:52.549 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.550 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.551 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.552 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.553 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.554 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.555 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.556 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.557 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.558 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.559 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.560 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.561 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.562 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.563 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.564 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.565 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.566 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.567 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.568 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.569 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.570 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.571 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.572 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.573 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.574 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.575 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-08-20T06:43:52.576 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.577 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.578 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.579 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.580 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.581 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.582 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.583 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.584 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.585 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.586 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.587 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.588 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.589 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-08-20T06:43:52.590 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.591 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.592 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.593 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.594 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.595 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.596 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.597 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.598 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.599 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.600 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.601 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.602 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.603 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.604 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.605 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.606 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.607 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.608 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.609 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.610 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.611 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.612 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.613 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.614 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.615 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.616 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.617 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.618 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.619 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.620 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.621 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.622 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-08-20T06:43:52.623 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.624 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.625 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.626 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.627 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.630 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.631 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.632 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.633 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.634 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.635 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.636 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.637 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.638 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.639 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.640 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.641 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.642 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.643 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.644 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.645 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.646 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.647 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.648 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.649 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.650 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.651 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.652 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.653 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:52.653 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-08-20T06:43:52.653 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.655 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.655 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-08-20T06:43:52.655 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T06:43:52.655 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.655 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-08-20T06:43:52.656 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.657 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.658 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.659 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.660 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.661 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.662 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.663 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.664 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.665 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.666 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.667 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.668 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.669 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.670 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.671 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.672 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.673 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.674 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.675 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.676 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.677 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.678 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.679 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.680 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.681 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.682 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.683 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.684 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-08-20T06:43:52.685 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.686 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.687 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:52.688 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.689 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.690 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.691 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.692 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.693 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.694 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.695 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.696 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.697 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.698 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.699 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-08-20T06:43:52.700 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.701 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.702 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.703 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-08-20T06:43:52.704 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.705 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-08-20T06:43:52.706 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.707 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.708 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.709 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.710 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.711 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.712 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.713 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.714 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.715 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.716 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.717 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.718 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.719 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.720 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.721 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.722 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.723 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.724 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.725 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.726 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.727 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.728 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-08-20T06:43:52.729 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.730 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.731 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.732 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.733 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.734 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.735 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.736 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.737 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.738 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.739 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.740 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.741 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.742 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.743 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.744 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.745 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.746 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.747 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.748 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.749 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.750 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-08-20T06:43:52.751 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.752 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.753 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.754 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.755 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.756 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.757 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.758 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-08-20T06:43:52.759 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.760 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.761 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.762 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.763 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smb", 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "internal_store_backend": { 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "internal_store_backend", 2024-08-20T06:43:52.764 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.765 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.766 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.767 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.768 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.769 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.770 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.771 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.772 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.773 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.774 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "update_orchestration": { 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "update_orchestration", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.775 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.776 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.777 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.778 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.779 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.780 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.781 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.782 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.783 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.784 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.785 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.786 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.787 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.788 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.789 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.790 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.791 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.792 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.793 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.794 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.795 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.796 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.797 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.798 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.799 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.800 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.801 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.802 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.803 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.804 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.805 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T06:43:52.806 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.807 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.808 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.809 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.810 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.811 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.812 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.813 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.814 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.815 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.816 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.817 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.818 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.819 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-08-20T06:43:52.820 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.821 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.822 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.823 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.824 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-08-20T06:43:52.825 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.826 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.827 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.828 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-08-20T06:43:52.829 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.830 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.831 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.832 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.833 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.834 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.835 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.836 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.837 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.838 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.839 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.840 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.841 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.842 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.843 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.844 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.845 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-08-20T06:43:52.846 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.847 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.848 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.849 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.850 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.851 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.852 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.853 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.854 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.857 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.858 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-08-20T06:43:52.859 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.860 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.861 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.862 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.863 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.864 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.865 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.866 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.867 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.868 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.869 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T06:43:52.870 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.871 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.872 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.873 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.874 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.875 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.876 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.877 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.878 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.879 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.880 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.881 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.882 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.883 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.884 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.885 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.886 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.887 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.888 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.889 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.890 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "modules": [ 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard", 2024-08-20T06:43:52.891 INFO:teuthology.orchestra.run.smithi107.stdout: "iostat", 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "nfs", 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "restful" 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "available_modules": [ 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "alerts", 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.892 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.893 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.894 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.895 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.896 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.897 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.898 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.899 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_destination": { 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_destination", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.900 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email address to send alerts to", 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_from_name": { 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_from_name", 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.901 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "Ceph", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Email From: name", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_host": { 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_host", 2024-08-20T06:43:52.902 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP server", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_password": { 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_password", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Password to authenticate with", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.904 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_port": { 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_port", 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.905 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "465", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP port", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_sender": { 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_sender", 2024-08-20T06:43:52.906 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SMTP envelope sender", 2024-08-20T06:43:52.907 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_ssl": { 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_ssl", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.908 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "smtp_user": { 2024-08-20T06:43:52.909 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smtp_user", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "User to authenticate as", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.910 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.911 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.912 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "balancer", 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "active": { 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "active", 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.913 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.914 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_time": { 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_time", 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0000", 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.915 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "begin_weekday": { 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "begin_weekday", 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.916 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.917 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_max_iterations": { 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_max_iterations", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "25", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "250", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T06:43:52.918 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_metrics": { 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_metrics", 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T06:43:52.919 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "crush_compat_step": { 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crush_compat_step", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.920 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.5", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0.001", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "0.999", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "aggressiveness of optimization", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "end_time": { 2024-08-20T06:43:52.921 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_time", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2359", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.922 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "end_weekday": { 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "end_weekday", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "0", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "6", 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.923 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.924 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.925 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.926 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.927 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.928 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.929 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "min_score": { 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_score", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.930 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:52.931 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "upmap", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "crush-compat", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "none", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "read", 2024-08-20T06:43:52.932 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap-read" 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Balancer mode", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_ids": { 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_ids", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.933 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.934 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.935 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:52.936 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.937 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_deviation": { 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_deviation", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T06:43:52.938 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "upmap_max_optimizations": { 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "upmap_max_optimizations", 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.939 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:52.940 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm", 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_down_multiplier": { 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_down_multiplier", 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.941 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.942 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_refresh_rate": { 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_refresh_rate", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "20", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.943 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "agent_starting_port": { 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "agent_starting_port", 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.944 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4721", 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.945 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_ptrace": { 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_ptrace", 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.946 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.947 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_interval": { 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_interval", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.948 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "autotune_memory_target_ratio": { 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.949 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0.7", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm_log_destination": { 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cephadm_log_destination", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.950 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "file", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "file,syslog", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "syslog" 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.951 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "cgroups_split": { 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cgroups_split", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.952 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "config_checks_enabled": { 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_checks_enabled", 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.953 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "config_dashboard": { 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_dashboard", 2024-08-20T06:43:52.954 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.955 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_alertmanager": { 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_alertmanager", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.956 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_base": { 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_base", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.957 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Container image name, without the tag", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_elasticsearch": { 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_elasticsearch", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.958 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "elasticsearch container image", 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_grafana": { 2024-08-20T06:43:52.959 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_grafana", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.960 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_haproxy": { 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_haproxy", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "HAproxy container image", 2024-08-20T06:43:52.961 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_agent": { 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_agent", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.962 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger agent container image", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_collector": { 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_collector", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.963 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger collector container image", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_jaeger_query": { 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_jaeger_query", 2024-08-20T06:43:52.964 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Jaeger query container image", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.965 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_keepalived": { 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_keepalived", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Keepalived container image", 2024-08-20T06:43:52.966 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_loki": { 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_loki", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.967 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Loki container image", 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nginx": { 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nginx", 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.968 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nginx container image", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_node_exporter": { 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_node_exporter", 2024-08-20T06:43:52.969 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.970 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_nvmeof": { 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_nvmeof", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.971 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Nvme-of container image", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_prometheus": { 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_prometheus", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.972 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Prometheus container image", 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.973 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_promtail": { 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_promtail", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Promtail container image", 2024-08-20T06:43:52.974 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_samba": { 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_samba", 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.975 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Samba/SMB container image", 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.976 INFO:teuthology.orchestra.run.smithi107.stdout: "container_image_snmp_gateway": { 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_image_snmp_gateway", 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T06:43:52.977 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "SNMP Gateway container image", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "container_init": { 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "container_init", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.978 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.979 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "daemon_cache_timeout": { 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "daemon_cache_timeout", 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.980 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.981 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_cephadm_command_timeout": { 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "900", 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.982 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.983 INFO:teuthology.orchestra.run.smithi107.stdout: "default_registry": { 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "default_registry", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "docker.io", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.984 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "device_cache_timeout": { 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_cache_timeout", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1800", 2024-08-20T06:43:52.985 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache device inventory", 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "device_enhanced_scan": { 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_enhanced_scan", 2024-08-20T06:43:52.986 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.987 INFO:teuthology.orchestra.run.smithi107.stdout: "facts_cache_timeout": { 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "facts_cache_timeout", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "seconds to cache host facts data", 2024-08-20T06:43:52.988 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "grafana_dashboards_path": { 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "grafana_dashboards_path", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.989 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "host_check_interval": { 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "host_check_interval", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:52.990 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to perform a host check", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "hw_monitoring": { 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hw_monitoring", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.991 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "inventory_list_all": { 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "inventory_list_all", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.992 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:52.993 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.994 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.995 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "log_refresh_metadata": { 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_refresh_metadata", 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.996 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.997 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:52.998 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:52.999 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.000 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.001 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T06:43:53.002 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "*", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.003 INFO:teuthology.orchestra.run.smithi107.stdout: "max_count_per_host": { 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_count_per_host", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of daemons per service per host", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.004 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max_osd_draining_count": { 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_osd_draining_count", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.005 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "migration_current": { 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "migration_current", 2024-08-20T06:43:53.006 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "internal - do not modify", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.007 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "mode": { 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mode", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "root", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.008 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm-package", 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "root" 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.009 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "oob_default_addr": { 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "oob_default_addr", 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "169.254.1.1", 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.010 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "prometheus_alerts_path": { 2024-08-20T06:43:53.011 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus_alerts_path", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.012 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_insecure": { 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_insecure", 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.013 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.014 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_password": { 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_password", 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.015 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_url": { 2024-08-20T06:43:53.016 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_url", 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.017 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "registry_username": { 2024-08-20T06:43:53.018 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "registry_username", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.019 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "secure_monitoring_stack": { 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secure_monitoring_stack", 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.020 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "service_discovery_port": { 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "service_discovery_port", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.021 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8765", 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "cephadm service discovery port", 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.022 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.023 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "ssh_config_file": { 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssh_config_file", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.024 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "use_agent": { 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_agent", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.025 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "use_repo_digest": { 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "use_repo_digest", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.026 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_failed_host_check": { 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_failed_host_check", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.027 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.028 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_daemons": { 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_daemons", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.029 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_on_stray_hosts": { 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_on_stray_hosts", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.030 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crash", 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.031 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.032 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.033 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.034 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.035 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.036 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.037 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "retain_interval": { 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retain_interval", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "31536000", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.038 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.039 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.040 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_recent_interval": { 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_recent_interval", 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1209600", 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.041 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.042 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dashboard", 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.043 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.044 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.045 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.046 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_ENABLED": { 2024-08-20T06:43:53.047 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.048 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.049 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T06:43:53.050 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.051 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.052 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.053 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.054 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.055 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.056 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.057 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.058 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.059 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T06:43:53.060 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.061 INFO:teuthology.orchestra.run.smithi107.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.062 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T06:43:53.063 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.066 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.067 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_URL": { 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_URL", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.068 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.069 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.070 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.071 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.072 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.073 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.074 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.075 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "[]", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T06:43:53.076 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "{}", 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.077 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.078 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.079 INFO:teuthology.orchestra.run.smithi107.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.080 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.081 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.082 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.083 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T06:43:53.084 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.085 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.086 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.087 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.088 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.089 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.090 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.091 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.092 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.093 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.094 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.095 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.096 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8", 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T06:43:53.097 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "45", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.098 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.099 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "admin", 2024-08-20T06:43:53.100 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.101 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T06:43:53.102 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.103 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.104 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.105 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.106 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.107 INFO:teuthology.orchestra.run.smithi107.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.108 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "cross_origin_url": { 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cross_origin_url", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.109 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "crt_file": { 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "crt_file", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.110 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "debug": { 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "debug", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.111 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Enable/disable debug options", 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.112 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "jwt_token_ttl": { 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "jwt_token_ttl", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "28800", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.113 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:53.114 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.115 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.116 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.117 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.118 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.119 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.120 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.121 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.122 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "motd": { 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "motd", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.123 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "The message of the day", 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect_resolve_ip_addr": { 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T06:43:53.124 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.125 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.126 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:53.127 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8080", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.128 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.129 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.130 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.131 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl_server_port": { 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl_server_port", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8443", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.132 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "redirect", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.133 INFO:teuthology.orchestra.run.smithi107.stdout: "redirect" 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.134 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.135 INFO:teuthology.orchestra.run.smithi107.stdout: "url_prefix": { 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url_prefix", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.136 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.137 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_monitoring": { 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_monitoring", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "monitor device health metrics", 2024-08-20T06:43:53.138 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.139 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.140 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.141 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.142 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.143 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.144 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "mark_out_threshold": { 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mark_out_threshold", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "2419200", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.145 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "pool_name": { 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pool_name", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "device_health_metrics", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.146 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "retention_period": { 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "retention_period", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.147 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15552000", 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long to retain device health metrics", 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.148 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_frequency": { 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_frequency", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.149 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "self_heal": { 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "self_heal", 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.150 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.151 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.152 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.153 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.154 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "warn_threshold": { 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "warn_threshold", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7257600", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.155 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.156 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "diskprediction_local", 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.157 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.158 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.159 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.160 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.161 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.162 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.163 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "predict_interval": { 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predict_interval", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.164 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "86400", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "predictor_model": { 2024-08-20T06:43:53.165 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "predictor_model", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "prophetstor", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.166 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "600", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.167 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.168 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "influx", 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": false, 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "influxdb python module not found", 2024-08-20T06:43:53.169 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "batch_size": { 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "batch_size", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5000", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.170 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "database": { 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "database", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "ceph", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.171 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "hostname": { 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "hostname", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.172 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server hostname", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.173 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "30", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "5", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.174 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.175 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.176 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.177 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.178 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.179 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.180 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "password": { 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "password", 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.181 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "password of InfluxDB server user", 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.182 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "port": { 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "port", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "8086", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "InfluxDB server port", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.183 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.184 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "ssl": { 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ssl", 2024-08-20T06:43:53.185 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "false", 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.186 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "threads": { 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threads", 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.187 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "32", 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.188 INFO:teuthology.orchestra.run.smithi107.stdout: "username": { 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "username", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.189 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "username of InfluxDB server user", 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "verify_ssl": { 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "verify_ssl", 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.190 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "true", 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.191 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "insights", 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.192 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.193 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.194 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.195 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.196 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.197 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.198 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.199 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.200 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.201 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.202 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.203 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.204 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.205 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.207 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "iostat", 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.208 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.209 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.210 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.211 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.212 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.213 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.214 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.215 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "k8sevents", 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_event_retention_days": { 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "ceph_event_retention_days", 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.216 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "7", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "config_check_secs": { 2024-08-20T06:43:53.217 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "config_check_secs", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "10", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.218 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.219 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.221 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.222 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.223 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.224 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.225 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.226 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.227 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.228 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "localpool", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "failure_domain": { 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "failure_domain", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "host", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.229 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "failure domain for any created local pool", 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.230 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.231 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.232 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.233 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.234 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.235 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.236 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.237 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.238 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.239 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "min_size": { 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "min_size", 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.240 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default min_size for any created local pool", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "num_rep": { 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "num_rep", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.241 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default replica count for any created local pool", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_num": { 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_num", 2024-08-20T06:43:53.242 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "128", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "prefix": { 2024-08-20T06:43:53.243 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prefix", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "name prefix for any created local pool", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.244 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.245 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "subtree": { 2024-08-20T06:43:53.246 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "subtree", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "rack", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.247 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mds_autoscaler", 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.248 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.249 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.250 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.251 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.252 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.253 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.254 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.255 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.256 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirroring", 2024-08-20T06:43:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.257 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.258 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.259 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.260 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.261 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.262 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.263 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.264 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.265 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.266 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.267 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "nfs", 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.268 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.269 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.271 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.272 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.273 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.274 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.275 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.276 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.277 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.278 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.279 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "fail_fs": { 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "fail_fs", 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.280 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.281 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.282 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.283 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.284 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.285 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.286 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.287 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator": { 2024-08-20T06:43:53.288 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "orchestrator", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "cephadm", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "rook", 2024-08-20T06:43:53.289 INFO:teuthology.orchestra.run.smithi107.stdout: "test_orchestrator" 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Orchestrator backend", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.290 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_perf_query", 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.291 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.292 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.293 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.294 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.295 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.296 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.297 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.298 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "osd_support", 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.299 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.300 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.301 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.302 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.303 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.304 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.305 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "pg_autoscaler", 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.306 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.307 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.308 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.309 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.310 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.311 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.312 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.313 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "threshold": { 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "threshold", 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.314 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "3.0", 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1.0", 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "scaling threshold", 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.315 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "progress", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_pg_recovery_event": { 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_pg_recovery_event", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.316 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.317 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.318 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.319 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.320 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.321 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.322 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "max_completed_events": { 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_completed_events", 2024-08-20T06:43:53.323 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "50", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "number of past completed events to remember", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "sleep_interval": { 2024-08-20T06:43:53.324 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sleep_interval", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "how long the module is going to sleep", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.325 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.326 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "prometheus", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "cache": { 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "cache", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.327 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "exclude_perf_counters": { 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "exclude_perf_counters", 2024-08-20T06:43:53.328 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T06:43:53.329 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.330 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.331 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.332 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.333 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.334 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools": { 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools", 2024-08-20T06:43:53.335 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T06:43:53.336 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "300", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.337 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "scrape_interval": { 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "scrape_interval", 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.338 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15.0", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.339 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "::", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:53.340 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "9283", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.341 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "stale_cache_strategy": { 2024-08-20T06:43:53.342 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stale_cache_strategy", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "log", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_behaviour": { 2024-08-20T06:43:53.343 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_behaviour", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "default", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "default", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "error" 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.344 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "standby_error_status_code": { 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "standby_error_status_code", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "500", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "400", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "599", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.345 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.346 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.347 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.348 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.349 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.350 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.351 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_snap_create": { 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_snap_create", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.352 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "mirror_snapshot_schedule": { 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.353 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.354 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "trash_purge_schedule": { 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "trash_purge_schedule", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.355 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.356 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "restful", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enable_auth": { 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enable_auth", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.357 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "key_file": { 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "key_file", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.358 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.359 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.360 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.361 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.362 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.363 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "server_addr": { 2024-08-20T06:43:53.364 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_addr", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "server_port": { 2024-08-20T06:43:53.365 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "server_port", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.366 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.367 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rgw", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.368 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.369 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.370 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.371 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.372 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T06:43:53.373 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "5", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.374 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.375 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rook", 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.376 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.377 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.378 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.379 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.380 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.381 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.382 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "storage_class": { 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "storage_class", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "local", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.383 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "selftest", 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.384 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.385 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.386 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.387 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.388 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.389 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "roption1": { 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption1", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.390 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "roption2": { 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "roption2", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "xyz", 2024-08-20T06:43:53.391 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption1": { 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption1", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.392 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption2": { 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption2", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.393 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption3": { 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption3", 2024-08-20T06:43:53.394 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "float", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption4": { 2024-08-20T06:43:53.395 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption4", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption5": { 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption5", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.397 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption6": { 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption6", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.398 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "rwoption7": { 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rwoption7", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "1", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "42", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.399 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.400 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "testkey": { 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testkey", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.401 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "testlkey": { 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testlkey", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.402 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "testnewline": { 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "testnewline", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.403 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "smb", 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.404 INFO:teuthology.orchestra.run.smithi107.stdout: "internal_store_backend": { 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "internal_store_backend", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.405 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.406 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.407 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.408 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.409 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.410 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.411 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "update_orchestration": { 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "update_orchestration", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T06:43:53.412 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snap_schedule", 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "allow_m_granularity": { 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "allow_m_granularity", 2024-08-20T06:43:53.413 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "dump_on_update": { 2024-08-20T06:43:53.414 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "dump_on_update", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "dump database to debug log on update", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.415 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.416 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.417 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.418 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.419 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.420 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.421 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "stats", 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.422 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.423 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.424 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.425 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.426 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.427 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "status", 2024-08-20T06:43:53.428 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.429 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.430 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.431 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.432 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.433 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.434 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.435 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telegraf", 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "address": { 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "address", 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.436 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:53.437 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "15", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.438 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.439 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.440 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.441 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.442 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.443 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.444 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "telemetry", 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_basic": { 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_basic", 2024-08-20T06:43:53.445 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_crash": { 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_crash", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.447 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_device": { 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_device", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.448 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_ident": { 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_ident", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.449 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "channel_perf": { 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "channel_perf", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.450 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "contact": { 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "contact", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.451 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "description": { 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "description", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.452 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "device_url": { 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "device_url", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.453 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "enabled": { 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "enabled", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.454 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:53.455 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "24", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "8", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "last_opt_revision": { 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "last_opt_revision", 2024-08-20T06:43:53.456 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "1", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.457 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard": { 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.458 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "leaderboard_description": { 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "leaderboard_description", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.459 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.460 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.461 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.462 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.463 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.464 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.465 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "organization": { 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "organization", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.466 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "proxy": { 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "proxy", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.467 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.468 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "url": { 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "url", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.469 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "test_orchestrator", 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.470 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.471 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.472 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.474 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.475 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.476 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.477 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.478 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.479 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.480 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.481 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.482 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.483 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "max_concurrent_clones": { 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "max_concurrent_clones", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.484 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "4", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "periodic_async_work": { 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "periodic_async_work", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.485 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Periodically check for async work", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_delay": { 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_delay", 2024-08-20T06:43:53.486 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.487 INFO:teuthology.orchestra.run.smithi107.stdout: "snapshot_clone_no_wait": { 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "True", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.488 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.489 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix", 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "can_run": true, 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "error_string": "", 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "module_options": { 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "discovery_interval": { 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "discovery_interval", 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "uint", 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.490 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "100", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "identifier": { 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "identifier", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.491 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "interval": { 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "interval", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "secs", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.492 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "60", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "log_level": { 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_level", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.493 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.494 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster": { 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.495 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_cluster_level": { 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_cluster_level", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "info", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [ 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "", 2024-08-20T06:43:53.496 INFO:teuthology.orchestra.run.smithi107.stdout: "critical", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "debug", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "error", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "info", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "warning" 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "log_to_file": { 2024-08-20T06:43:53.497 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "log_to_file", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "bool", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "False", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.498 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "sqlite3_killpoint": { 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "sqlite3_killpoint", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "dev", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 1, 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "0", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.499 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_host": { 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_host", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.500 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_port": { 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_port", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "int", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "10051", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.501 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "zabbix_sender": { 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "zabbix_sender", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "str", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "level": "advanced", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "flags": 0, 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "min": "", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "max": "", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "enum_allowed": [], 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "desc": "", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "long_desc": "", 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "tags": [], 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: "see_also": [] 2024-08-20T06:43:53.502 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "services": { 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "dashboard": "https://172.21.15.107:8443/" 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "always_on_modules": { 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "octopus": [ 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-08-20T06:43:53.503 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "pacific": [ 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-08-20T06:43:53.504 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "quincy": [ 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-08-20T06:43:53.505 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "reef": [ 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-08-20T06:43:53.506 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: ], 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "squid": [ 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "balancer", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "crash", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "devicehealth", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "orchestrator", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "pg_autoscaler", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "progress", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "rbd_support", 2024-08-20T06:43:53.507 INFO:teuthology.orchestra.run.smithi107.stdout: "status", 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "telemetry", 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "volumes" 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "last_failure_osd_epoch": 3, 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "active_clients": [ 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "devicehealth", 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.508 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 142658058 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "libcephsqlite", 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1268797056 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.509 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "rbd_support", 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 3678314558 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.510 INFO:teuthology.orchestra.run.smithi107.stdout: "name": "volumes", 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "addrvec": [ 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: { 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "type": "v2", 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "addr": "172.21.15.107:0", 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: "nonce": 1990235887 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-08-20T06:43:53.511 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 msgr2=0x7fa17c0b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 0x7fa17c0b6090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa184000c00 tx=0x7fa184031400 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 msgr2=0x7fa194199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c970 tx=0x7fa18800ce30 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 shutdown_connections 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fa17c0b3bd0 0x7fa17c0b6090 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa194103ba0 0x7fa194199d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa194102db0 0x7fa194199140 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 --2- 172.21.15.107:0/1719977820 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fa1941023f0 0x7fa194198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 >> 172.21.15.107:0/1719977820 conn(0x7fa1940fdf40 msgr2=0x7fa194106be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:43:53.513 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 shutdown_connections 2024-08-20T06:43:53.514 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:43:51.717+0000 7fa198c85640 1 -- 172.21.15.107:0/1719977820 wait complete. 2024-08-20T06:43:53.520 DEBUG:tasks.check_counter:Getting stats from a 2024-08-20T06:43:53.520 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-20T06:43:53.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2428243732' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:43:53.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:52 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1719977820' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T06:43:53.521 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:53 smithi107 ceph-mon[18495]: pgmap v538: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-08-20T06:43:53.522 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:52 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2428243732' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:43:53.522 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:52 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1719977820' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T06:43:53.522 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:52 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2428243732' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T06:43:53.522 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:52 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1719977820' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T06:43:53.522 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:53 smithi183 ceph-mon[22204]: pgmap v538: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-08-20T06:43:53.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:53 smithi193 ceph-mon[21944]: pgmap v538: 97 pgs: 97 active+clean; 177 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 427 op/s 2024-08-20T06:43:53.771 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 2960, 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-08-20T06:43:54.443 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003112808, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000037816, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002168265, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.004009889, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001336629 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:54.444 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 155, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 150, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 801712, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 91088, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-08-20T06:43:54.445 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.023247664, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.010888075, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.017566980, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 149, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.005538475, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000037170 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:54.446 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 801712, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 91088 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 113, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 160, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 99712, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.009487987, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.007291401, 2024-08-20T06:43:54.447 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001277478, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 111, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003662652, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000032996 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:43:54.448 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 99712 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:43:54.449 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:43:54.450 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:43:54.451 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148967, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 29, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-08-20T06:43:54.452 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-08-20T06:43:54.453 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 156, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 779829, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 156, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 156, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 779829, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:54.454 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:54.455 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:55.136 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-20T06:43:55.136 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-20T06:43:55.136 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-08-20T06:43:55.136 DEBUG:tasks.check_counter:Getting stats from d 2024-08-20T06:43:55.136 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-20T06:43:55.384 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:55.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:55 smithi183 ceph-mon[22204]: pgmap v539: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-08-20T06:43:55.505 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:55 smithi107 ceph-mon[18495]: pgmap v539: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-08-20T06:43:55.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:55 smithi193 ceph-mon[21944]: pgmap v539: 97 pgs: 97 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 425 op/s 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 2960, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-08-20T06:43:55.902 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003120802, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000039066, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002211772, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.004069020, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001356340 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:55.903 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 117, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 160, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 100416, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.009432292, 2024-08-20T06:43:55.904 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.007357164, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001098312, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 115, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.004274444, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000037169 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:43:55.905 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 100416 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 160, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 155, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 803072, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 94288, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.025385751, 2024-08-20T06:43:55.906 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011298794, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.020612275, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 154, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.006332694, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000041121 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:55.907 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 803072, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 94288 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:43:55.908 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-08-20T06:43:55.909 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:43:55.910 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149007, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:43:55.911 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-08-20T06:43:55.912 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 161, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 780479, 2024-08-20T06:43:55.913 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 161, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 161, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 780479, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:55.914 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:56.652 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-20T06:43:56.652 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-20T06:43:56.652 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-08-20T06:43:56.652 DEBUG:tasks.check_counter:Getting stats from g 2024-08-20T06:43:56.653 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-20T06:43:56.897 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:57.276 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:57 smithi107 ceph-mon[18495]: pgmap v540: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 841 KiB/s wr, 309 op/s 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 157, 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 152, 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 802256, 2024-08-20T06:43:57.385 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 92368, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.026089153, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.010771961, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.021624614, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 151, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.006385403, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000042287 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.386 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 802256, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 92368 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 5, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 116, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 3120, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 100336, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-08-20T06:43:57.387 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.010964344, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.006948951, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.003235187, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 115, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.007357927, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000063981 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:57.388 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 3120, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 100336 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 0, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 2, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 0, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 0, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-08-20T06:43:57.389 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.000323164, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000000000, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.000258009, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:57.390 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.391 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:43:57.392 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-08-20T06:43:57.393 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:43:57.394 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 148984, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-08-20T06:43:57.395 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:57.396 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 158, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 780089, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 158, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 158, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 780089, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:57.397 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:57.398 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:57.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:57.398 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:57.398 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:57.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:57 smithi183 ceph-mon[22204]: pgmap v540: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 841 KiB/s wr, 309 op/s 2024-08-20T06:43:57.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:57 smithi193 ceph-mon[21944]: pgmap v540: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 860 KiB/s rd, 841 KiB/s wr, 309 op/s 2024-08-20T06:43:58.044 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-08-20T06:43:58.045 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-08-20T06:43:58.045 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-08-20T06:43:58.045 DEBUG:tasks.check_counter:Getting stats from j 2024-08-20T06:43:58.045 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-20T06:43:58.291 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:43:58.809 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 4, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 4, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 2960, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 448, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 4, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 0, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.003025783, 2024-08-20T06:43:58.810 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.000034552, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.002214347, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 3, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.003994144, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.001331381 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.811 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 162, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 157, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 803616, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 95568, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 2, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 1, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.025448258, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.011798850, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.019652583, 2024-08-20T06:43:58.812 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 156, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.005995137, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000038430 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 803616, 2024-08-20T06:43:58.813 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 95568 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_messages": 1, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages": 118, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_bytes": 160, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_bytes": 100592, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_created_connections": 3, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_active_connections": 2, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_total_time": 0.010782645, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_send_time": 0.008155608, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_recv_time": 0.001653270, 2024-08-20T06:43:58.814 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 116, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.005535047, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000047715 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "msgr_send_encrypted_bytes": 100592 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.815 INFO:teuthology.orchestra.run.smithi107.stdout: "cct": { 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "total_workers": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "unhealthy_workers": 0 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "mempool": { 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bloom_filter_items": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:43:58.816 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_extent_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_blob_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:43:58.817 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_txc_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluestore_writing_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:43:58.818 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_bytes": 149007, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_anon_items": 30, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "buffer_meta_items": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_items": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_mapbl_items": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osd_pglog_items": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_bytes": 0, 2024-08-20T06:43:58.819 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_items": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_bytes": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "pgmap_items": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_bytes": 928, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "mds_co_items": 2, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_bytes": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_1_items": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_bytes": 0, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "unittest_2_items": 0 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: }, 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:43:58.820 INFO:teuthology.orchestra.run.smithi107.stdout: "val": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "max": 104857600, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "get_started": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "get": 163, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "get_sum": 780739, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_fail": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "get_or_fail_success": 163, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "take": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "take_sum": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "put": 163, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "put_sum": 780739, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "wait": { 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "avgcount": 0, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "sum": 0.000000000, 2024-08-20T06:43:58.821 INFO:teuthology.orchestra.run.smithi107.stdout: "avgtime": 0.000000000 2024-08-20T06:43:58.822 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:58.822 INFO:teuthology.orchestra.run.smithi107.stdout: } 2024-08-20T06:43:58.822 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-08-20T06:43:59.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:43:59 smithi183 ceph-mon[22204]: pgmap v541: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 796 KiB/s wr, 267 op/s 2024-08-20T06:43:59.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:43:59 smithi193 ceph-mon[21944]: pgmap v541: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 796 KiB/s wr, 267 op/s 2024-08-20T06:43:59.583 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:43:59 smithi107 ceph-mon[18495]: pgmap v541: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 796 KiB/s wr, 267 op/s 2024-08-20T06:43:59.614 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-20T06:43:59.614 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-20T06:43:59.614 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-08-20T06:43:59.614 DEBUG:tasks.check_counter:Getting stats from b 2024-08-20T06:43:59.618 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-20T06:43:59.875 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:44:00.446 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-08-20T06:44:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:00.446 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 29751, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 29943, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 11206718, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 207210152, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 68, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 4, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 2.694590436, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 1.436754980, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.575001847, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.600920171, 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:00.447 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 29942, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.782530628, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000126328 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7860, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001111896, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000141 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 905648, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 111840 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:00.448 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 822045, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 298934, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 289173698, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 300129217, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 24, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 23.734477913, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 11.277024908, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 16.787361195, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.761454883, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 298933, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 9.943962118, 2024-08-20T06:44:00.449 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000033264 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1013579, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.094940338, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000093 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 55600, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 316640 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.450 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 34370, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 34601, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 12220969, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 166472140, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 10, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 3.056844216, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 1.693339712, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.597849484, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.656026194, 2024-08-20T06:44:00.451 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 34599, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 5.997424768, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000173340 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 9519, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001367077, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000143 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.452 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 41002, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 67.247990881, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001640114 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:00.453 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 29777, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 4.190675297, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000140735 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 120353, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 109679, 2024-08-20T06:44:00.454 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 109679, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 508.687496271, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.004637966 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 10674, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 5, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 2, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 1327, 2024-08-20T06:44:00.455 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 69, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 27, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 6, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 7, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 14, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 18, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 633, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 2, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 2, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 4, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 181755, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 135199, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 10612, 2024-08-20T06:44:00.456 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 1, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 5, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 6, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 416, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 34, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 4, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 47976, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-08-20T06:44:00.457 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 47976, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 484, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 9, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 475, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 47492, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 27007, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 46643, 2024-08-20T06:44:00.458 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 6, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 664857, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 18476, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 367, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 104025 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 1, 2024-08-20T06:44:00.459 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 11, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 2, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 18195, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 18193, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:00.460 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 9, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 8, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 69, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:00.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 351815, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 247511, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 245801, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 106014, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 1710, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 422, 2024-08-20T06:44:00.462 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 294, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 292, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 130, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 9, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 2, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 343064397, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 491612867, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8527, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 30.499044678, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003576761 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.463 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 28, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 18443, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 18415, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 23, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 711, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 688, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 27, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 45272, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 45245, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 2, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 18420, 2024-08-20T06:44:00.464 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 18418, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 123964, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 371056640 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 130603, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 1333, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 120353, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 23, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 2676, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 18299, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 54.359796183, 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002970642 2024-08-20T06:44:00.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 881, 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.442141562, 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000501863 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:00.466 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 42759, 2024-08-20T06:44:00.467 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 110.780208043, 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002590804 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.469 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 110, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.344012453, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003127385 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.471 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 15040, 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 42.464729138, 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002823452 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-08-20T06:44:00.472 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1015, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.321394165, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001301866 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 17917, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 274.293937182, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.015309144 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 104, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 7.796929676, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.074970477 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.474 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001775037, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001775037 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:00.475 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 8, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.019629451, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002453681 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001748049, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000582683 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 13542, 2024-08-20T06:44:00.477 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 16.861195332, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001245103 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 3, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 2, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 29044, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 29044, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 426, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.479 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:00.480 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:00.481 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:00.482 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 308024, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 120, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 88, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 1, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:00.483 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 156, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 115350, 2024-08-20T06:44:00.484 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 916, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 91369, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 512893879, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 37, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 91369, 2024-08-20T06:44:00.485 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 91369, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 309.945218238, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.003392236 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 91369, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 118347 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 91369, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 48686, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 42683, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-08-20T06:44:00.486 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 667, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 11803, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 694, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 9212, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 381, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 69, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 19485, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:00.487 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 47992, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 23606, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 6, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-08-20T06:44:00.488 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-08-20T06:44:00.489 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 73, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 3394, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 14, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 1024 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-08-20T06:44:00.490 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 18, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 18777, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 18193, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.491 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 886162, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 232823123, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 886162, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 886162, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 232823123, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.492 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:00.493 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 91369, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 514731868, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 91369, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 514731868, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.494 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 91369, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 91369, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 91369, 2024-08-20T06:44:00.495 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 91369, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 18193, 2024-08-20T06:44:00.496 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 1838077, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 18193, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 685, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1838077, 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:00.497 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55a4447786e0": { 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:00.498 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 351815, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 487418563, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 351815, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 8527, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 487418563, 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:00.499 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:00.500 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-08-20T06:44:01.200 INFO:tasks.check_counter:Daemon mds.b mds.exported=6 2024-08-20T06:44:01.200 INFO:tasks.check_counter:Daemon mds.b mds.imported=34 2024-08-20T06:44:01.200 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=69 2024-08-20T06:44:01.201 DEBUG:tasks.check_counter:Getting stats from e 2024-08-20T06:44:01.201 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-20T06:44:01.455 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:44:01.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:01 smithi183 ceph-mon[22204]: pgmap v542: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 453 KiB/s wr, 196 op/s 2024-08-20T06:44:01.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:01 smithi193 ceph-mon[21944]: pgmap v542: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 453 KiB/s wr, 196 op/s 2024-08-20T06:44:01.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:01 smithi107 ceph-mon[18495]: pgmap v542: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 453 KiB/s wr, 196 op/s 2024-08-20T06:44:01.912 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-08-20T06:44:01.912 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 658, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 660, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 4367223, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 209308, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 6, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.056716559, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.026616204, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.013669997, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.011852439, 2024-08-20T06:44:01.913 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 658, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.016677845, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000025346 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:01.914 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 611, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 595, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1052110, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 233305, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.063921855, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.027006215, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.031276760, 2024-08-20T06:44:01.915 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.010903708, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 595, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.021878045, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000036769 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 859584, 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 96560 2024-08-20T06:44:01.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 532, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 648, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 178576, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 449946, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.064111556, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.031245014, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.012956042, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.014127158, 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:01.917 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 646, 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.027553622, 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000042652 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 281088 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2761, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.061748054, 2024-08-20T06:44:01.919 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000022364 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 0, 2024-08-20T06:44:01.920 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 353, 2024-08-20T06:44:01.921 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 353, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 353, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 1, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 0, 2024-08-20T06:44:01.922 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 0, 2024-08-20T06:44:01.923 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:01.926 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 0, 2024-08-20T06:44:01.926 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 0, 2024-08-20T06:44:01.926 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-08-20T06:44:01.926 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 0, 2024-08-20T06:44:01.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 0 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:01.928 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:01.929 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:01.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4687, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-08-20T06:44:01.931 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 7, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 8475044, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 8474445, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.932 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 4687 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 8, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 370, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 362, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 6, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 37, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 31, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 353, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1046, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 693, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 0, 2024-08-20T06:44:01.933 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 53096, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 252493824 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.934 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-08-20T06:44:01.935 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.937 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.939 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-08-20T06:44:01.939 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.939 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.940 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.941 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.942 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:01.943 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.945 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.946 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-08-20T06:44:01.947 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 2, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 2336, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:01.948 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:01.949 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:01.950 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 189383, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 50, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:01.951 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 156, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 266112, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 1485, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:01.952 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 1619, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 21, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 3, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 1619, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1619, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.685980600, 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000423706 2024-08-20T06:44:01.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1619, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1620 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 1619, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1619, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 0, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 958, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 0, 2024-08-20T06:44:01.954 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 659, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 0, 2024-08-20T06:44:01.955 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-08-20T06:44:01.956 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-08-20T06:44:01.957 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 73, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 5, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 5, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-08-20T06:44:01.958 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 0, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 2, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 0 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 0, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-08-20T06:44:01.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:01.960 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 1797, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5424142, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 1797, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1797, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5424142, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1619, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4282613, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1619, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4282613, 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:01.962 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:01.963 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 1619, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 1619, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 1619, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 1619, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:01.964 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.965 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x55a04676c960": { 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 0, 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:01.966 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:01.967 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-08-20T06:44:02.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:02 smithi183 ceph-mon[22204]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:44:02.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:02 smithi193 ceph-mon[21944]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:44:02.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:02 smithi107 ceph-mon[18495]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T06:44:02.670 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-20T06:44:02.670 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-20T06:44:02.670 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-08-20T06:44:02.671 DEBUG:tasks.check_counter:Getting stats from h 2024-08-20T06:44:02.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-20T06:44:02.915 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:44:03.357 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:03 smithi183 ceph-mon[22204]: pgmap v543: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 429 KiB/s wr, 132 op/s 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 5403, 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 5132, 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 1873285, 2024-08-20T06:44:03.459 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 5929674, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 9, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 3, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.377895397, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.190120368, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.182215181, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.032705610, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 5132, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.107974568, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000021039 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:03.460 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7763, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.001157644, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000149 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 788, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 903, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 206223, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 865991, 2024-08-20T06:44:03.461 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 15, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 6, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.062890169, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.028653238, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.014808993, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.014676738, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 901, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.042028529, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000046646 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 21, 2024-08-20T06:44:03.462 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000002493, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000118 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 284480 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 14716, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 11396, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 5650022, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 6407765, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 23, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 7, 2024-08-20T06:44:03.463 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.886169127, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.421754514, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.608780992, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.010109316, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 11394, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.209935844, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000018425 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 24286, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.003340941, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000137 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.464 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 870112, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 104896 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 1, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-MDSRank": { 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2816, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.892048546, 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000316778 2024-08-20T06:44:03.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "queue_len": 0, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "complete_latency": { 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 643, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.047645843, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000074099 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "mds": { 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "request": 7901, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "reply": 7900, 2024-08-20T06:44:03.466 INFO:teuthology.orchestra.run.smithi183.stdout: "reply_latency": { 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 7900, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 19.575189226, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.002477872 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_reply": 0, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "forward": 1, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_complete": 0, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_fetch_keys": 0, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_commit": 29, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_split": 2, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_merge": 0, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes": 16, 2024-08-20T06:44:03.467 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_top": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_bottom": 16, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pin_tail": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_pinned": 13, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_expired": 14, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "inodes_with_caps": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "caps": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "subtrees": 3, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse": 17525, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_hit": 10466, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_forward": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_discover": 4, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:03.468 INFO:teuthology.orchestra.run.smithi183.stdout: "traverse_lock": 5, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "load_cent": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "q": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "exported": 16, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "exported_inodes": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "imported": 2, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "imported_inodes": 114, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_backtrace_fetch": 1113, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rfiles": 4, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rbytes": 234, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "root_rsnaps": 0, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_fetch": 1113, 2024-08-20T06:44:03.469 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_inodes": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "scrub_file_inodes": 1113, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_inode_file_caps": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_revoke": 341, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_grant": 1183, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:03.470 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps": 6087, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_caps_dirty": 292, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_cap_release": 12, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "process_request_cap_release": 1853 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_cache": { 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_try_discover": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_send_discover": 5, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "dir_handle_discover": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:03.471 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_created": 348, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_enqueued": 348, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "strays_migrated": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_started": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "recovery_completed": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_exportdir": 16, 2024-08-20T06:44:03.472 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_flush": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragmentdir": 2, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "ireq_inodestats": 0 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_log": { 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evlrg": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evadd": 4687, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evex": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evtrm": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "ev": 4687, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evexg": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "evexd": 0, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "segadd": 7, 2024-08-20T06:44:03.473 INFO:teuthology.orchestra.run.smithi183.stdout: "segex": 0, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "segtrm": 0, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "seg": 7, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "segmjr": 2, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "segexg": 0, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "segexd": 0, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "expos": 4194304, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "wrpos": 8475044, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "rdpos": 0, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "jlat": { 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 905, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 1.686077486, 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001863069 2024-08-20T06:44:03.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "replayed": 0 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_mem": { 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ino": 20, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ino+": 375, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ino-": 355, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dir": 19, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dir+": 43, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dir-": 24, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dn": 16, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dn+": 1096, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "dn-": 1080, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "cap": 0, 2024-08-20T06:44:03.475 INFO:teuthology.orchestra.run.smithi183.stdout: "cap+": 353, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "cap-": 353, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "rss": 60672, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "heap": 271376384 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_server": { 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_client_request": 9665, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "dispatch_server_request": 2, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_request": 7901, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_client_session": 11, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "handle_peer_request": 605, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "req_create_latency": { 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 292, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8.396418347, 2024-08-20T06:44:03.476 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.028754857 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getattr_latency": { 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 1, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000186495, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000186495 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "req_link_latency": { 2024-08-20T06:44:03.477 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookup_latency": { 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 6689, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2.108007264, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000315145 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupino_latency": { 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupname_latency": { 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.479 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "req_lssnap_latency": { 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.480 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mkdir_latency": { 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mknod_latency": { 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "req_mksnap_latency": { 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "req_open_latency": { 2024-08-20T06:44:03.482 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 260, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.074904640, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000288094 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "req_readdir_latency": { 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 27, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.013976358, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000517642 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rename_latency": { 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 292, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 8.600581191, 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.029454045 2024-08-20T06:44:03.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmdir_latency": { 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.484 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setattr_latency": { 2024-08-20T06:44:03.485 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setlayout_latency": { 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_setxattr_latency": { 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "req_symlink_latency": { 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.487 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "req_unlink_latency": { 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 339, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.381114931, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001124232 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_sessions": { 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "session_count": 1, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "session_add": 2, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "session_remove": 1, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_open": 1, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "sessions_stale": 0, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "total_load": 2200, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "average_load": 2200, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "avg_session_uptime": 424, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-08-20T06:44:03.489 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:03.490 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:03.491 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 280379, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 96, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:03.492 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 156, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-08-20T06:44:03.493 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 80813, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 356, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "objecter": { 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_active": 0, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_laggy": 0, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send": 2606, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_send_bytes": 4635735, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_resend": 28, 2024-08-20T06:44:03.494 INFO:teuthology.orchestra.run.smithi183.stdout: "op_reply": 2606, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "op_latency": { 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2606, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 3.970887671, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001523748 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "op_inflight": 0, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "oplen_avg": { 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 2606, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 2809 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.495 INFO:teuthology.orchestra.run.smithi183.stdout: "op": 2606, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "op_r": 1128, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "op_w": 1478, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "op_rmw": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "op_pg": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_stat": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_create": 10, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_read": 15, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_write": 920, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writefull": 33, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_writesame": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_append": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_zero": 0, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_truncate": 2, 2024-08-20T06:44:03.496 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_delete": 361, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_mapext": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_getxattr": 1113, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_setxattr": 20, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_call": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_watch": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_notify": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls": 0, 2024-08-20T06:44:03.497 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "osdop_other": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_active": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_send": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_resend": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "linger_ping": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_active": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_send": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolop_resend": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_active": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_send": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "poolstat_resend": 0, 2024-08-20T06:44:03.498 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_active": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_send": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "statfs_resend": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "command_active": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "command_send": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "command_resend": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "map_epoch": 73, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "map_full": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "map_inc": 2, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_sessions": 12, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_open": 12, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_session_close": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_laggy": 0, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_wr": 261, 2024-08-20T06:44:03.499 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_rd": 0, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_del": 74 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "oft": { 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_objs": 1, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_updates": 0, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "omap_total_removes": 0 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "purge_queue": { 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_ops_high_water": 10, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing": 0, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executing_high_water": 10, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed_ops": 348, 2024-08-20T06:44:03.500 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_executed": 348, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 20903, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 5834252, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 20903, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 20903, 2024-08-20T06:44:03.501 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 5834252, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:03.502 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2606, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 4670883, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2606, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4670883, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:03.503 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 1024, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 2606, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 2606, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 2606, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 2606, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.504 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 348, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 35148, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 348, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 15, 2024-08-20T06:44:03.505 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 35148, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-write_buf_throttle-0x556c6f1226e0": { 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 3758096384, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 4687, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 4280740, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 4687, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 905, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 4280740, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "trackedop": { 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "slow_ops_count": 0 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:03.507 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-08-20T06:44:03.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:03 smithi193 ceph-mon[21944]: pgmap v543: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 429 KiB/s wr, 132 op/s 2024-08-20T06:44:03.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:03 smithi107 ceph-mon[18495]: pgmap v543: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 429 KiB/s wr, 132 op/s 2024-08-20T06:44:04.266 INFO:tasks.check_counter:Daemon mds.h mds.exported=16 2024-08-20T06:44:04.266 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-08-20T06:44:04.266 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=2 2024-08-20T06:44:04.266 DEBUG:tasks.check_counter:Getting stats from k 2024-08-20T06:44:04.267 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-20T06:44:04.513 INFO:teuthology.orchestra.run.smithi183.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 164, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 159, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 804160, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 96848, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 4, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 0, 2024-08-20T06:44:05.003 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.026628284, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.011117126, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.022577205, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 158, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.006927286, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000043843 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.004 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 804160, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 96848 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 1, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 120, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 160, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 100944, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 3, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 2, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.007910890, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.005942704, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.000918204, 2024-08-20T06:44:05.005 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 118, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.005669613, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000048047 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 100944 2024-08-20T06:44:05.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_messages": 4, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages": 4, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_bytes": 2960, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_bytes": 448, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_created_connections": 2, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_active_connections": 1, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_total_time": 0.002266658, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_send_time": 0.000049720, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_recv_time": 0.002219487, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 3, 2024-08-20T06:44:05.007 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.004012392, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.001337464 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_recv_encrypted_bytes": 2960, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "cct": { 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "total_workers": 0, 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: "unhealthy_workers": 0 2024-08-20T06:44:05.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "mempool": { 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:05.009 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:05.010 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_items": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:05.011 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_bytes": 148967, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_anon_items": 29, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_bytes": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_items": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:05.012 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_bytes": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_items": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "pgmap_items": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_bytes": 928, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "mds_co_items": 2, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_1_items": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "unittest_2_items": 0 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:05.013 INFO:teuthology.orchestra.run.smithi183.stdout: "val": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "max": 104857600, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "get_started": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "get": 165, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "get_sum": 780999, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "get_or_fail_success": 165, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "take": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "take_sum": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "put": 165, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "put_sum": 780999, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "wait": { 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "avgcount": 0, 2024-08-20T06:44:05.014 INFO:teuthology.orchestra.run.smithi183.stdout: "sum": 0.000000000, 2024-08-20T06:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: "avgtime": 0.000000000 2024-08-20T06:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-08-20T06:44:05.015 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-08-20T06:44:05.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:05 smithi193 ceph-mon[21944]: pgmap v544: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 72 op/s 2024-08-20T06:44:05.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:05 smithi107 ceph-mon[18495]: pgmap v544: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 72 op/s 2024-08-20T06:44:05.622 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:05 smithi183 ceph-mon[22204]: pgmap v544: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 82 KiB/s wr, 72 op/s 2024-08-20T06:44:05.648 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-20T06:44:05.648 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-20T06:44:05.648 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-08-20T06:44:05.648 DEBUG:tasks.check_counter:Getting stats from c 2024-08-20T06:44:05.649 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-20T06:44:05.907 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1402, 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1385, 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 153089451, 2024-08-20T06:44:06.386 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 484366, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.222131975, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.061590162, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.128987445, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.029965986, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1385, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.053689817, 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000038765 2024-08-20T06:44:06.387 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 859744, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 96720 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 507, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 623, 2024-08-20T06:44:06.388 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 159518164, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 443678, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 7, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.108901918, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.022867362, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.073069935, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.009185775, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 621, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.022089919, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000035571 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.389 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 282704 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 601, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 603, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 176317881, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 191125, 2024-08-20T06:44:06.390 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.123398832, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.023620914, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.082395125, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.013304064, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 601, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.020325045, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000033818 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.391 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:06.392 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3664, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.183386710, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000050050 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.393 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-08-20T06:44:06.394 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 18276, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 2912, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 15364, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 14116, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 2, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-08-20T06:44:06.395 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:06.396 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-08-20T06:44:06.397 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:06.398 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:06.399 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-08-20T06:44:06.400 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 351815, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 130, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 30, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 343064397, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 491612867, 2024-08-20T06:44:06.401 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 491608566, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 351815 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 25, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 18425, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 18400, 2024-08-20T06:44:06.402 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 20, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 706, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 686, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 18276, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 45163, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 26887, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 92788, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 317771776 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-08-20T06:44:06.403 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.404 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-08-20T06:44:06.405 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.406 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.407 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.408 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.409 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.410 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:06.411 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.412 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.413 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-08-20T06:44:06.414 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.415 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 3, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 2, 2024-08-20T06:44:06.416 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2339, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:06.417 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:06.418 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:06.419 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 239488, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 80, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:06.420 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 156, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 12509269, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 73602, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:06.421 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 2326, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 26, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 4, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 2326, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2326, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.219965190, 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000954413 2024-08-20T06:44:06.422 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2326, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2327 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 2326, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 2326, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 1328, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-08-20T06:44:06.423 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 995, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 2, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:06.424 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-08-20T06:44:06.425 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 73, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-08-20T06:44:06.426 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 2, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 11, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 11, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-08-20T06:44:06.427 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:06.428 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 2506, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 488687873, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 2506, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2506, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 488687873, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.429 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2326, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 487427742, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2326, 2024-08-20T06:44:06.430 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 487427742, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:06.431 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 2326, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 2326, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 2326, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 2326, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.432 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:06.433 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x56283f5b2960": { 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:06.434 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:06.435 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-08-20T06:44:07.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:07 smithi193 ceph-mon[21944]: pgmap v545: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:07.315 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-20T06:44:07.315 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-20T06:44:07.315 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-08-20T06:44:07.315 DEBUG:tasks.check_counter:Getting stats from f 2024-08-20T06:44:07.315 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-20T06:44:07.561 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:44:07.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:07 smithi107 ceph-mon[18495]: pgmap v545: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:07.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:07 smithi183 ceph-mon[22204]: pgmap v545: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:08.025 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-08-20T06:44:08.025 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:08.025 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1676, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 1678, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 6657921, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 532975, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 6, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.146979383, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.064778248, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.033215575, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.039371658, 2024-08-20T06:44:08.026 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1676, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.057336131, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000034210 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:08.027 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 3, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 120, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 716, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 283578, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 3, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.009215250, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.006515955, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.001459759, 2024-08-20T06:44:08.028 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000073378, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 118, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.007714006, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000065372 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.029 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 282880 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 183, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 167, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 860124, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 97518, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 3, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.031737076, 2024-08-20T06:44:08.030 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.012873113, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.026697654, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000022678, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 167, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.013429935, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000080418 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.031 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 859856, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 97200 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:08.032 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 2917, 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.064528908, 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000022121 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.033 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 0, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 0, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.034 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 511, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 0, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 511, 2024-08-20T06:44:08.035 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 511, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 1, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:08.036 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 0, 2024-08-20T06:44:08.037 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T06:44:08.038 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 0 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 0, 2024-08-20T06:44:08.039 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-08-20T06:44:08.040 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-08-20T06:44:08.041 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 6618, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 8, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-08-20T06:44:08.042 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 10382272, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 10381134, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 6618 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 10, 2024-08-20T06:44:08.043 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 535, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 525, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 8, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 46, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 38, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 511, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 1428, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 917, 2024-08-20T06:44:08.044 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 52056, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 253546496 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 0, 2024-08-20T06:44:08.045 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:08.046 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.047 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-08-20T06:44:08.048 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.049 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.050 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.051 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.052 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.053 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.054 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.055 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.056 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-08-20T06:44:08.057 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-08-20T06:44:08.058 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 2, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 0, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 0, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 2336, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:08.059 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:08.060 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:08.061 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 181343, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 47, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-08-20T06:44:08.062 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 156, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 380560, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 2139, 2024-08-20T06:44:08.063 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 1679, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 21, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 3, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 1679, 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-08-20T06:44:08.064 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1679, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.727988610, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000433584 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1679, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1680 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 1679, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1679, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 0, 2024-08-20T06:44:08.065 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 970, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 707, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:08.066 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-08-20T06:44:08.067 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-08-20T06:44:08.068 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 73, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 2, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 4, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 4, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 0, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 2, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 0 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.069 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 0, 2024-08-20T06:44:08.070 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 1858, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 7339452, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 1858, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1858, 2024-08-20T06:44:08.071 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 7339452, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:08.072 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1679, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6187973, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1679, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6187973, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:08.073 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 1679, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 1679, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1679, 2024-08-20T06:44:08.074 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 1679, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:08.075 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.076 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x5594412fc960": { 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 0, 2024-08-20T06:44:08.077 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:08.078 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-08-20T06:44:08.777 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-08-20T06:44:08.777 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-08-20T06:44:08.777 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-08-20T06:44:08.777 DEBUG:tasks.check_counter:Getting stats from i 2024-08-20T06:44:08.778 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-20T06:44:09.022 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:44:09.434 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:09 smithi193 ceph-mon[21944]: pgmap v546: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 29426, 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 21822, 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 9458293, 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 14378582, 2024-08-20T06:44:09.537 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 16, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 1.976281287, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.910244977, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 1.433453643, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.039703767, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 21818, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.461376192, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000021146 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 46717, 2024-08-20T06:44:09.538 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.007409550, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000158 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 5859, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 5614, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 3057994, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 1955995, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 9, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 5, 2024-08-20T06:44:09.539 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.392216368, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.183319583, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.254620062, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.014464993, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 5615, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.116992868, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000020835 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 9548, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001454483, 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000152 2024-08-20T06:44:09.540 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 871712, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 108400 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 837, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 951, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 272130, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 879042, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 22, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 6, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.074902291, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.033705855, 2024-08-20T06:44:09.541 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.020689353, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.015495739, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 950, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.040259008, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000042377 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 21, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000002402, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000114 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.542 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 4048, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 285840 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 1, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-MDSRank": { 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3527, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 1.070942749, 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000303641 2024-08-20T06:44:09.543 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "finisher-PurgeQueue": { 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "queue_len": 0, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "complete_latency": { 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1130, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.048355571, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000042792 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "mds": { 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "request": 16299, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "reply": 16298, 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "reply_latency": { 2024-08-20T06:44:09.544 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 16298, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 26.044905932, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001598043 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_reply": 0, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "forward": 1, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_complete": 0, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_fetch_keys": 0, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_commit": 29, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_split": 2, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_merge": 0, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes": 19, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_top": 2, 2024-08-20T06:44:09.545 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_bottom": 3, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pin_tail": 14, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_pinned": 14, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_expired": 24, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "inodes_with_caps": 4, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "caps": 4, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "subtrees": 3, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse": 34839, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_hit": 11306, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_forward": 0, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_discover": 5, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_dir_fetch": 0, 2024-08-20T06:44:09.546 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_remote_ino": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "traverse_lock": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "load_cent": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "q": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "exported": 18, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "exported_inodes": 4, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "imported": 4, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "imported_inodes": 302, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_dir_fetch": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_backtrace_fetch": 1377, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "openino_peer_discover": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rfiles": 4, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rbytes": 234, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "root_rsnaps": 0, 2024-08-20T06:44:09.547 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_fetch": 1377, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_set_tag": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_inotable_repaired": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_inodes": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "scrub_file_inodes": 1377, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_inode_file_caps": 2, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_revoke": 403, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_grant": 1524, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T06:44:09.548 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps": 11162, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_caps_dirty": 353, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_cap_release": 37, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "process_request_cap_release": 2366 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_cache": { 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_update_receipt": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_try_discover": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_send_discover": 6, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "dir_handle_discover": 1, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_delayed": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "num_strays_enqueuing": 0, 2024-08-20T06:44:09.549 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_created": 504, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_enqueued": 504, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_reintegrated": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "strays_migrated": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_processing": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_enqueued": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "num_recovering_prioritized": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_started": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "recovery_completed": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_path": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_quiesce_inode": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_exportdir": 18, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_flush": 0, 2024-08-20T06:44:09.550 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragmentdir": 2, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_fragstats": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "ireq_inodestats": 0 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_log": { 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evlrg": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evadd": 6618, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evex": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evtrm": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "ev": 6618, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evexg": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "evexd": 0, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "segadd": 8, 2024-08-20T06:44:09.551 INFO:teuthology.orchestra.run.smithi193.stdout: "segex": 0, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "segtrm": 0, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "seg": 8, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "segmjr": 2, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "segexg": 0, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "segexd": 0, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "expos": 4194304, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "wrpos": 10382272, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "rdpos": 0, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "jlat": { 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 1126, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 2.216896246, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001968824 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.552 INFO:teuthology.orchestra.run.smithi193.stdout: "replayed": 0 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_mem": { 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "ino": 23, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "ino+": 536, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "ino-": 513, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir": 22, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir+": 48, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dir-": 26, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dn": 19, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dn+": 1564, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "dn-": 1545, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "cap": 4, 2024-08-20T06:44:09.553 INFO:teuthology.orchestra.run.smithi193.stdout: "cap+": 512, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "cap-": 508, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "rss": 58724, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "heap": 270262272 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_server": { 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_client_request": 18460, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "dispatch_server_request": 5, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_request": 16299, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_client_session": 11, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "handle_peer_request": 744, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "req_create_latency": { 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 353, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 10.530436541, 2024-08-20T06:44:09.554 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.029831264 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getattr_latency": { 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 4, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.001737850, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000434462 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getfilelock_latency": { 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "req_link_latency": { 2024-08-20T06:44:09.555 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookup_latency": { 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 14974, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.061733594, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000271252 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookuphash_latency": { 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.556 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupino_latency": { 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupname_latency": { 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupparent_latency": { 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.557 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lookupsnap_latency": { 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "req_lssnap_latency": { 2024-08-20T06:44:09.558 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mkdir_latency": { 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mknod_latency": { 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.559 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_mksnap_latency": { 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_open_latency": { 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 135, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.041641265, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000308453 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "req_readdir_latency": { 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 28, 2024-08-20T06:44:09.560 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.017828119, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000636718 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rename_latency": { 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 356, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 10.951106762, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.030761535 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_renamesnap_latency": { 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "req_snapdiff_latency": { 2024-08-20T06:44:09.561 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmdir_latency": { 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmsnap_latency": { 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.562 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "req_rmxattr_latency": { 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setattr_latency": { 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setdirlayout_latency": { 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.563 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setfilelock_latency": { 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setlayout_latency": { 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "req_setxattr_latency": { 2024-08-20T06:44:09.564 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "req_symlink_latency": { 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "req_unlink_latency": { 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 448, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.440421801, 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000983084 2024-08-20T06:44:09.565 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_revoke_eviction": 0, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "cap_acquisition_throttle": 0, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "req_getvxattr_latency": { 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_sessions": { 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "session_count": 1, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "session_add": 2, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "session_remove": 1, 2024-08-20T06:44:09.566 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_open": 1, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "sessions_stale": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "total_load": 4432, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "average_load": 4432, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "avg_session_uptime": 432, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "mdthresh_evicted": 0 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:09.567 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:09.568 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:09.569 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 285351, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 111, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:09.570 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 7592, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 156, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 94989, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 420, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "objecter": { 2024-08-20T06:44:09.571 INFO:teuthology.orchestra.run.smithi193.stdout: "op_active": 0, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_laggy": 0, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send": 3408, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_send_bytes": 6582090, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_resend": 29, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_reply": 3408, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_latency": { 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3408, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 4.508929225, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.001323042 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "op_inflight": 0, 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "oplen_avg": { 2024-08-20T06:44:09.572 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 3408, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 3680 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "op": 3408, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "op_r": 1435, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "op_w": 1973, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "op_rmw": 0, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "op_pg": 0, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_stat": 0, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_create": 10, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_read": 58, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_write": 1184, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writefull": 63, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_writesame": 0, 2024-08-20T06:44:09.573 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_append": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_zero": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_truncate": 2, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_delete": 517, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_mapext": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_sparse_read": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_clonerange": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_getxattr": 1377, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_setxattr": 20, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_cmpxattr": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_rmxattr": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_resetxattrs": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_call": 0, 2024-08-20T06:44:09.574 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_watch": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_notify": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_pgls_filter": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "osdop_other": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_active": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_send": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_resend": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "linger_ping": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_active": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_send": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "poolop_resend": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_active": 0, 2024-08-20T06:44:09.575 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_send": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "poolstat_resend": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_active": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_send": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "statfs_resend": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "command_active": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "command_send": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "command_resend": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "map_epoch": 73, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "map_full": 0, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "map_inc": 2, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_sessions": 12, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_open": 12, 2024-08-20T06:44:09.576 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_session_close": 0, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_laggy": 0, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_wr": 336, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_rd": 0, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_del": 113 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "oft": { 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_objs": 1, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_kv_pairs": 0, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_updates": 0, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "omap_total_removes": 0 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "purge_queue": { 2024-08-20T06:44:09.577 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops": 0, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_ops_high_water": 10, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing": 0, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executing_high_water": 10, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed_ops": 504, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_executed": 504, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "pq_item_in_journal": 0 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 36118, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 9523744, 2024-08-20T06:44:09.578 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 36118, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 36118, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 9523744, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_bytes": { 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:09.579 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3408, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 6632994, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3408, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6632994, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.580 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-objecter_ops": { 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 1024, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 0, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 3408, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 3408, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 3408, 2024-08-20T06:44:09.581 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 3408, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle": { 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 504, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 50904, 2024-08-20T06:44:09.582 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 504, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 58, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 50904, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-write_buf_throttle-0x55ef902e26e0": { 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:09.583 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 3758096384, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 6618, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 6187968, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 6618, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 1126, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 6187968, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:09.584 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.585 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:09.585 INFO:teuthology.orchestra.run.smithi193.stdout: "trackedop": { 2024-08-20T06:44:09.585 INFO:teuthology.orchestra.run.smithi193.stdout: "slow_ops_count": 0 2024-08-20T06:44:09.585 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:09.585 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-08-20T06:44:09.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:09 smithi107 ceph-mon[18495]: pgmap v546: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:09.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:09 smithi183 ceph-mon[22204]: pgmap v546: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T06:44:10.402 INFO:tasks.check_counter:Daemon mds.i mds.exported=18 2024-08-20T06:44:10.403 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-08-20T06:44:10.403 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=2 2024-08-20T06:44:10.403 DEBUG:tasks.check_counter:Getting stats from l 2024-08-20T06:44:10.403 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-20T06:44:10.649 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T06:44:11.160 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 0, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 2, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 0, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 0, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 4, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 0, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.000996877, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.000000000, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.000131455, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:11.161 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.162 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 1, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 122, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 160, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 101296, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 3, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 2, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.009046337, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.006998154, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.000793112, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 120, 2024-08-20T06:44:11.163 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.006015146, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000050126 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 101296 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_messages": 170, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages": 163, 2024-08-20T06:44:11.164 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_bytes": 807664, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_bytes": 98576, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_created_connections": 2, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_active_connections": 1, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_total_time": 0.031680743, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_send_time": 0.013219880, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_recv_time": 0.026456091, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 163, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.013328732, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000081771 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.165 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_handle_ack_lat": { 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_recv_encrypted_bytes": 807664, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "msgr_send_encrypted_bytes": 98576 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "cct": { 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "total_workers": 0, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "unhealthy_workers": 0 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "mempool": { 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_bytes": 0, 2024-08-20T06:44:11.166 INFO:teuthology.orchestra.run.smithi193.stdout: "bloom_filter_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_alloc_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_data_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_other_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_bytes": 0, 2024-08-20T06:44:11.167 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_extent_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_blob_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_fsck_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_txc_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_bytes": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluestore_writing_items": 0, 2024-08-20T06:44:11.168 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_reader_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "bluefs_file_writer_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_bytes": 148967, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_anon_items": 29, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "buffer_meta_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_mapbl_items": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_bytes": 0, 2024-08-20T06:44:11.169 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_pglog_items": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_bytes": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_items": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "osdmap_mapping_items": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_bytes": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "pgmap_items": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_bytes": 928, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_co_items": 2, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_bytes": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_1_items": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_bytes": 0, 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: "unittest_2_items": 0 2024-08-20T06:44:11.170 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "val": 0, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "max": 104857600, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_started": 0, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get": 167, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_sum": 781259, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_fail": 0, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "get_or_fail_success": 167, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "take": 0, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "take_sum": 0, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "put": 167, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "put_sum": 781259, 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "wait": { 2024-08-20T06:44:11.171 INFO:teuthology.orchestra.run.smithi193.stdout: "avgcount": 0, 2024-08-20T06:44:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "sum": 0.000000000, 2024-08-20T06:44:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: "avgtime": 0.000000000 2024-08-20T06:44:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:11.172 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-08-20T06:44:11.172 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-08-20T06:44:11.304 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:11 smithi193 ceph-mon[21944]: pgmap v547: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-20T06:44:11.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:11 smithi183 ceph-mon[22204]: pgmap v547: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-20T06:44:11.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:11 smithi107 ceph-mon[18495]: pgmap v547: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 7 op/s 2024-08-20T06:44:11.767 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-20T06:44:11.767 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-20T06:44:11.767 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-08-20T06:44:11.768 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-08-20T06:44:11.777 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-08-20T06:44:11.777 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:44:11.777 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T06:44:11.798 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:44:11.798 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T06:44:11.858 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T06:44:12.153 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 -- 172.21.15.107:0/4210621712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc1073a0 msgr2=0x7ff5bc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/4210621712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc1077e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009900 tx=0x7ff5a402a410 comp rx=0 tx=0).stop 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 -- 172.21.15.107:0/4210621712 shutdown_connections 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/4210621712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc107d20 0x7ff5bc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/4210621712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc1077e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/4210621712 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc101990 0x7ff5bc101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 -- 172.21.15.107:0/4210621712 >> 172.21.15.107:0/4210621712 conn(0x7ff5bc0fd400 msgr2=0x7ff5bc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 -- 172.21.15.107:0/4210621712 shutdown_connections 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.604+0000 7ff5c0c45640 1 -- 172.21.15.107:0/4210621712 wait complete. 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.605+0000 7ff5c0c45640 1 Processor -- start 2024-08-20T06:44:12.644 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.605+0000 7ff5c0c45640 1 -- start start 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.605+0000 7ff5c0c45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.605+0000 7ff5c0c45640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.605+0000 7ff5c0c45640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc107d20 0x7ff5bc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5c0c45640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc1118d0 con 0x7ff5bc101990 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5c0c45640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc111750 con 0x7ff5bc107d20 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5c0c45640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc111a50 con 0x7ff5bc1073a0 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58766/0 (socket says 172.21.15.107:58766) 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 -- 172.21.15.107:0/2775235699 learned_addr learned my addr 172.21.15.107:0/2775235699 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5bad9d640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc107d20 0x7ff5bc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5b1d9b640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:12.645 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 -- 172.21.15.107:0/2775235699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 msgr2=0x7ff5bc19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 -- 172.21.15.107:0/2775235699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc107d20 msgr2=0x7ff5bc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc107d20 0x7ff5bc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 -- 172.21.15.107:0/2775235699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1a5220 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5b1d9b640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5ba59c640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b4a0 tx=0x7ff5ac00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac014070 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff5ac010aa0 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc1a5510 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac017b70 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.606+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1a5ad0 con 0x7ff5bc101990 2024-08-20T06:44:12.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.607+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc101df0 con 0x7ff5bc101990 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.609+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff5ac017070 con 0x7ff5bc101990 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.610+0000 7ff5b3fff640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 0x7ff5a80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.610+0000 7ff5b1d9b640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 0x7ff5a80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.610+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7ff5ac0e3490 con 0x7ff5bc101990 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.611+0000 7ff5b1d9b640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 0x7ff5a80b6090 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4032d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:12.647 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.613+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff5ac0aaae0 con 0x7ff5bc101990 2024-08-20T06:44:12.800 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.799+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff5bc06abe0 con 0x7ff5bc101990 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.800+0000 7ff5b3fff640 1 -- 172.21.15.107:0/2775235699 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v73) ==== 81+0+598 (secure 0 0 0) 0x7ff5bc06abe0 con 0x7ff5bc101990 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1518535638 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2483940460 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3373920322 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3759743791 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3013940704 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/235474616 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:12.801 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/235474616 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:12.802 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/484222332 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:12.802 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3759743791 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:12.802 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/343023724 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:12.802 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-08-20T06:44:12.803 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 msgr2=0x7ff5a80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 0x7ff5a80b6090 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4000c00 tx=0x7ff5a4032d20 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 msgr2=0x7ff5bc199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b4a0 tx=0x7ff5ac00b960 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 shutdown_connections 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff5bc107d20 0x7ff5bc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5a80b3bd0 0x7ff5a80b6090 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5bc1073a0 0x7ff5bc19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 --2- 172.21.15.107:0/2775235699 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff5bc101990 0x7ff5bc199e20 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.802+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 >> 172.21.15.107:0/2775235699 conn(0x7ff5bc0fd400 msgr2=0x7ff5bc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.803+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 shutdown_connections 2024-08-20T06:44:12.804 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:12.803+0000 7ff5c0c45640 1 -- 172.21.15.107:0/2775235699 wait complete. 2024-08-20T06:44:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:13 smithi107 ceph-mon[18495]: pgmap v548: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:13.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:13 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2775235699' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:13 smithi193 ceph-mon[21944]: pgmap v548: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:13.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:13 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2775235699' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:13 smithi183 ceph-mon[22204]: pgmap v548: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:13.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:13 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2775235699' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:14.129 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-08-20T06:44:14.130 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T06:44:14.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T06:44:14.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:44:14.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:14 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.446 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:44:14.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:14 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T06:44:14.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:14 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.941+0000 7fb9320e2640 1 -- 172.21.15.107:0/3434517116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c10ce30 msgr2=0x7fb92c10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 --2- 172.21.15.107:0/3434517116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c10ce30 0x7fb92c10f290 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb924009ca0 tx=0x7fb92402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 -- 172.21.15.107:0/3434517116 shutdown_connections 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 --2- 172.21.15.107:0/3434517116 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c10ce30 0x7fb92c10f290 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 --2- 172.21.15.107:0/3434517116 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c1042f0 0x7fb92c10c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.953 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 --2- 172.21.15.107:0/3434517116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1039d0 0x7fb92c103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 -- 172.21.15.107:0/3434517116 >> 172.21.15.107:0/3434517116 conn(0x7fb92c0fd410 msgr2=0x7fb92c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 -- 172.21.15.107:0/3434517116 shutdown_connections 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 -- 172.21.15.107:0/3434517116 wait complete. 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 Processor -- start 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.942+0000 7fb9320e2640 1 -- start start 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 0x7fb92c1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c10ce30 0x7fb92c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c1129e0 con 0x7fb92c1039d0 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c112860 con 0x7fb92c10ce30 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fb92c112b60 con 0x7fb92c1042f0 2024-08-20T06:44:14.954 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:58794/0 (socket says 172.21.15.107:58794) 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 -- 172.21.15.107:0/4240622508 learned_addr learned my addr 172.21.15.107:0/4240622508 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92affd640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 0x7fb92c1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92bfff640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c10ce30 0x7fb92c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 -- 172.21.15.107:0/4240622508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 msgr2=0x7fb92c1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 0x7fb92c1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 -- 172.21.15.107:0/4240622508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c10ce30 msgr2=0x7fb92c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c10ce30 0x7fb92c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 -- 172.21.15.107:0/4240622508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92c19f230 con 0x7fb92c1039d0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92affd640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 0x7fb92c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb92b7fe640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00b4a0 tx=0x7fb91c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c014070 con 0x7fb92c1039d0 2024-08-20T06:44:14.955 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fb91c010aa0 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92c19f520 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.943+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017b70 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.944+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb92c1a5be0 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.944+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb92c103db0 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.946+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fb91c017070 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.947+0000 7fb928ff9640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 0x7fb9140b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.947+0000 7fb92affd640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 0x7fb9140b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.947+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fb91c0e33c0 con 0x7fb92c1039d0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.947+0000 7fb92affd640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 0x7fb9140b6090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ffad0 tx=0x7fb92000a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:14.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:14.949+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb91c0aaa10 con 0x7fb92c1039d0 2024-08-20T06:44:15.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.135+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb92c06ae90 con 0x7fb92c1039d0 2024-08-20T06:44:15.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.135+0000 7fb928ff9640 1 -- 172.21.15.107:0/4240622508 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v73) ==== 81+0+598 (secure 0 0 0) 0x7fb91c0aa310 con 0x7fb92c1039d0 2024-08-20T06:44:15.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/1518535638 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:15.136 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/2483940460 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3373920322 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/3759743791 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/3013940704 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6801/235474616 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/235474616 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/484222332 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:6800/3759743791 2024-08-21T06:28:01.208606+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stdout:172.21.15.107:0/343023724 2024-08-21T06:27:38.122407+0000 2024-08-20T06:44:15.137 INFO:teuthology.orchestra.run.smithi107.stderr:listed 10 entries 2024-08-20T06:44:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.137+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 msgr2=0x7fb9140b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.137+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 0x7fb9140b6090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0ffad0 tx=0x7fb92000a680 comp rx=0 tx=0).stop 2024-08-20T06:44:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.137+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 msgr2=0x7fb92c199180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:15.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.137+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00b4a0 tx=0x7fb91c00b960 comp rx=0 tx=0).stop 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 shutdown_connections 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fb92c10ce30 0x7fb92c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fb9140b3bd0 0x7fb9140b6090 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c1042f0 0x7fb92c1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 --2- 172.21.15.107:0/4240622508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb92c1039d0 0x7fb92c199180 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 >> 172.21.15.107:0/4240622508 conn(0x7fb92c0fd410 msgr2=0x7fb92c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 shutdown_connections 2024-08-20T06:44:15.140 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:15.138+0000 7fb9320e2640 1 -- 172.21.15.107:0/4240622508 wait complete. 2024-08-20T06:44:15.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: pgmap v549: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Unable to set osd_memory_target on smithi193 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.558 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: Unable to set osd_memory_target on smithi107 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:44:15.559 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:15 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4240622508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:15.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: pgmap v549: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Unable to set osd_memory_target on smithi193 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: Unable to set osd_memory_target on smithi107 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:44:15.612 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:44:15.613 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:15 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4240622508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: pgmap v549: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi193 to 176.0M 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi183 to 176.0M 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Unable to set osd_memory_target on smithi193 to 184598220: error parsing value: Value '184598220' is below minimum 939524096 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Unable to set osd_memory_target on smithi183 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Adjusting osd_memory_target on smithi107 to 175.9M 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: Unable to set osd_memory_target on smithi107 to 184532992: error parsing value: Value '184532992' is below minimum 939524096 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' 2024-08-20T06:44:15.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:15 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4240622508' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T06:44:15.751 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi107.front.sepia.ceph.com... 2024-08-20T06:44:15.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:44:15.751 DEBUG:teuthology.orchestra.run.smithi107:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-08-20T06:44:15.779 INFO:teuthology.orchestra.run:waiting for 300 2024-08-20T06:44:15.780 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi107.stderr:ceph-fuse[70495]: fuse finished with error 0 and tester_r 0 2024-08-20T06:44:17.538 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:17 smithi183 ceph-mon[22204]: pgmap v550: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:17.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:17 smithi193 ceph-mon[21944]: pgmap v550: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:17.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:17 smithi107 ceph-mon[18495]: pgmap v550: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-08-20T06:44:19.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:19 smithi193 ceph-mon[21944]: pgmap v551: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-08-20T06:44:19.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:19 smithi107 ceph-mon[18495]: pgmap v551: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-08-20T06:44:19.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:19 smithi183 ceph-mon[22204]: pgmap v551: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-08-20T06:44:21.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:21 smithi193 ceph-mon[21944]: pgmap v552: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:44:21.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:21 smithi107 ceph-mon[18495]: pgmap v552: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:44:21.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:21 smithi183 ceph-mon[22204]: pgmap v552: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 597 B/s wr, 7 op/s 2024-08-20T06:44:21.882 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi107.front.sepia.ceph.com 2024-08-20T06:44:21.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:44:21.882 DEBUG:teuthology.orchestra.run.smithi107:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-20T06:44:21.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:44:21.902 DEBUG:teuthology.orchestra.run.smithi107:> ip netns list 2024-08-20T06:44:21.961 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-20T06:44:21.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:44:21.962 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T06:44:22.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T06:44:22.032 DEBUG:teuthology.orchestra.run.smithi107:> sudo ip link delete ceph-brx 2024-08-20T06:44:22.131 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-20T06:44:22.141 INFO:tasks.cephadm:Teardown begin 2024-08-20T06:44:22.141 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-20T06:44:22.142 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:44:22.395 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:22.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- 172.21.15.107:0/3807938053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c107d20 msgr2=0x7f7b6c10e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:22.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 --2- 172.21.15.107:0/3807938053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c107d20 0x7f7b6c10e520 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009ca0 tx=0x7f7b6402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.896 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- 172.21.15.107:0/3807938053 shutdown_connections 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 --2- 172.21.15.107:0/3807938053 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c107d20 0x7f7b6c10e520 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 --2- 172.21.15.107:0/3807938053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 --2- 172.21.15.107:0/3807938053 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c101990 0x7f7b6c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- 172.21.15.107:0/3807938053 >> 172.21.15.107:0/3807938053 conn(0x7f7b6c0fd400 msgr2=0x7f7b6c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- 172.21.15.107:0/3807938053 shutdown_connections 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- 172.21.15.107:0/3807938053 wait complete. 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 Processor -- start 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.884+0000 7f7b7138a640 1 -- start start 2024-08-20T06:44:22.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c101990 0x7f7b6c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 0x7f7b6c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c1118e0 con 0x7f7b6c1073a0 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c111760 con 0x7f7b6c107d20 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c111a60 con 0x7f7b6c101990 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57840/0 (socket says 172.21.15.107:57840) 2024-08-20T06:44:22.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 -- 172.21.15.107:0/4084858743 learned_addr learned my addr 172.21.15.107:0/4084858743 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b6b7fe640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 0x7f7b6c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b6affd640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c101990 0x7f7b6c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 -- 172.21.15.107:0/4084858743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c101990 msgr2=0x7f7b6c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c101990 0x7f7b6c199c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 -- 172.21.15.107:0/4084858743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 msgr2=0x7f7b6c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 0x7f7b6c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 -- 172.21.15.107:0/4084858743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c1a5030 con 0x7f7b6c1073a0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b6b7fe640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 0x7f7b6c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b627fc640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400b4a0 tx=0x7f7b5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54014070 con 0x7f7b6c1073a0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f7b54010aa0 con 0x7f7b6c1073a0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c1a5320 con 0x7f7b6c1073a0 2024-08-20T06:44:22.899 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b54017b70 con 0x7f7b6c1073a0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.885+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c1a58b0 con 0x7f7b6c1073a0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.886+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c101d70 con 0x7f7b6c1073a0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.889+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f7b54017070 con 0x7f7b6c1073a0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.889+0000 7f7b68ff9640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 0x7f7b580b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.889+0000 7f7b6affd640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 0x7f7b580b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.890+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f7b540e3380 con 0x7f7b6c1073a0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.890+0000 7f7b6affd640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 0x7f7b580b6110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00ba40 tx=0x7f7b5c005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:22.900 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:22.892+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7b540aa9d0 con 0x7f7b6c1073a0 2024-08-20T06:44:23.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.104+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b6c06abe0 con 0x7f7b6c1073a0 2024-08-20T06:44:23.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.105+0000 7f7b68ff9640 1 -- 172.21.15.107:0/4084858743 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11937 (secure 0 0 0) 0x7f7b6c06abe0 con 0x7f7b6c1073a0 2024-08-20T06:44:23.106 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:44:23.106 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":32,"btime":"2024-08-20T06:44:01:222186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:44:00.327442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:44:23.106 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-08-20T06:44:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 msgr2=0x7f7b580b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:23.108 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 0x7f7b580b6110 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00ba40 tx=0x7f7b5c005cc0 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 msgr2=0x7f7b6c19a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f7b5400b4a0 tx=0x7f7b5400b960 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 shutdown_connections 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f7b6c107d20 0x7f7b6c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7b6c1073a0 0x7f7b6c19a170 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f7b580b3c50 0x7f7b580b6110 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 --2- 172.21.15.107:0/4084858743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7b6c101990 0x7f7b6c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 >> 172.21.15.107:0/4084858743 conn(0x7f7b6c0fd400 msgr2=0x7f7b6c0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 shutdown_connections 2024-08-20T06:44:23.109 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:23.107+0000 7f7b7138a640 1 -- 172.21.15.107:0/4084858743 wait complete. 2024-08-20T06:44:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:23 smithi107 ceph-mon[18495]: pgmap v553: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:23.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:23 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4084858743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:23.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:23 smithi193 ceph-mon[21944]: pgmap v553: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:23.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:23 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4084858743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:23 smithi183 ceph-mon[22204]: pgmap v553: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:23.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:23 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4084858743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:23.838 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T06:44:24.085 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- 172.21.15.107:0/1180677293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 msgr2=0x7fe9fc101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 --2- 172.21.15.107:0/1180677293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc101d70 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009900 tx=0x7fe9ec02a410 comp rx=0 tx=0).stop 2024-08-20T06:44:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- 172.21.15.107:0/1180677293 shutdown_connections 2024-08-20T06:44:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 --2- 172.21.15.107:0/1180677293 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc107d20 0x7fe9fc10e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.580 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 --2- 172.21.15.107:0/1180677293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 --2- 172.21.15.107:0/1180677293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc101d70 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- 172.21.15.107:0/1180677293 >> 172.21.15.107:0/1180677293 conn(0x7fe9fc0fd400 msgr2=0x7fe9fc0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- 172.21.15.107:0/1180677293 shutdown_connections 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- 172.21.15.107:0/1180677293 wait complete. 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 Processor -- start 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.567+0000 7fea0451d640 1 -- start start 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 0x7fe9fc19a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc1118d0 con 0x7fe9fc101990 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc111750 con 0x7fe9fc1073a0 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc111a50 con 0x7fe9fc107d20 2024-08-20T06:44:24.581 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:57860/0 (socket says 172.21.15.107:57860) 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 -- 172.21.15.107:0/4025273429 learned_addr learned my addr 172.21.15.107:0/4025273429 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea01ab8640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 -- 172.21.15.107:0/4025273429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 msgr2=0x7fe9fc19a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea02aba640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 0x7fe9fc19a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 0x7fe9fc19a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 -- 172.21.15.107:0/4025273429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 msgr2=0x7fe9fc19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 -- 172.21.15.107:0/4025273429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc1a5220 con 0x7fe9fc101990 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea01ab8640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea02aba640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 0x7fe9fc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:44:24.582 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea022b9640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009a20 tx=0x7fe9ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec03f030 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc1a54b0 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7fe9ec0035f0 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.568+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec035070 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.569+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1a5a60 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.570+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc101df0 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.571+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7fe9ec003aa0 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.572+0000 7fe9f37fe640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 0x7fe9e80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.572+0000 7fea01ab8640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 0x7fe9e80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.572+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7fe9ec104630 con 0x7fe9fc101990 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.572+0000 7fea01ab8640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 0x7fe9e80b6090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc06b3b0 tx=0x7fe9f8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:24.583 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.575+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe9ec0cbc80 con 0x7fe9fc101990 2024-08-20T06:44:24.789 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.787+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9fc109570 con 0x7fe9fc101990 2024-08-20T06:44:24.790 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.788+0000 7fe9f37fe640 1 -- 172.21.15.107:0/4025273429 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11937 (secure 0 0 0) 0x7fe9ec0cb580 con 0x7fe9fc101990 2024-08-20T06:44:24.790 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:44:24.791 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":32,"btime":"2024-08-20T06:44:01:222186+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14691,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6835/3097875231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6834","nonce":3097875231},{"type":"v1","addr":"172.21.15.107:6835","nonce":3097875231}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14703,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6837/3157081894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6836","nonce":3157081894},{"type":"v1","addr":"172.21.15.107:6837","nonce":3157081894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14757,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6839/1902182493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6838","nonce":1902182493},{"type":"v1","addr":"172.21.15.107:6839","nonce":1902182493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14769,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.107:6841/1702418500","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6840","nonce":1702418500},{"type":"v1","addr":"172.21.15.107:6841","nonce":1702418500}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24508,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6833/154109335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":154109335},{"type":"v1","addr":"172.21.15.193:6833","nonce":154109335}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24512,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.183:6833/4048095343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6832","nonce":4048095343},{"type":"v1","addr":"172.21.15.183:6833","nonce":4048095343}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T06:35:00.478558+0000","modified":"2024-08-20T06:44:00.327442+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24572,"mds_1":24562,"mds_2":24548},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24533":{"gid":24533,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.183:6835/2773184516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6834","nonce":2773184516},{"type":"v1","addr":"172.21.15.183:6835","nonce":2773184516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24541":{"gid":24541,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6835/2974563037","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2974563037},{"type":"v1","addr":"172.21.15.193:6835","nonce":2974563037}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.193:6837/2519256232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":2519256232},{"type":"v1","addr":"172.21.15.193:6837","nonce":2519256232}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24548":{"gid":24548,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.183:6837/2933033317","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6836","nonce":2933033317},{"type":"v1","addr":"172.21.15.183:6837","nonce":2933033317}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24562":{"gid":24562,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6839/3581594961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6838","nonce":3581594961},{"type":"v1","addr":"172.21.15.193:6839","nonce":3581594961}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24572":{"gid":24572,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.183:6839/2078682144","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6838","nonce":2078682144},{"type":"v1","addr":"172.21.15.183:6839","nonce":2078682144}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24572,"qdb_cluster":[24548,24562,24572]},"id":1}]} 2024-08-20T06:44:24.793 INFO:teuthology.orchestra.run.smithi107.stderr:dumped fsmap epoch 32 2024-08-20T06:44:24.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 msgr2=0x7fe9e80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:24.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 0x7fe9e80b6090 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc06b3b0 tx=0x7fe9f8008040 comp rx=0 tx=0).stop 2024-08-20T06:44:24.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 msgr2=0x7fe9fc199e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:24.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009a20 tx=0x7fe9ec004270 comp rx=0 tx=0).stop 2024-08-20T06:44:24.794 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 shutdown_connections 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7fe9e80b3bd0 0x7fe9e80b6090 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe9fc107d20 0x7fe9fc19a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7fe9fc1073a0 0x7fe9fc19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 --2- 172.21.15.107:0/4025273429 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe9fc101990 0x7fe9fc199e40 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 >> 172.21.15.107:0/4025273429 conn(0x7fe9fc0fd400 msgr2=0x7fe9fc0ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:24.795 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 shutdown_connections 2024-08-20T06:44:24.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:24.790+0000 7fea0451d640 1 -- 172.21.15.107:0/4025273429 wait complete. 2024-08-20T06:44:25.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:25 smithi107 ceph-mon[18495]: pgmap v554: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-20T06:44:25.443 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:25 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4025273429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:25.471 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T06:44:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:25 smithi193 ceph-mon[21944]: pgmap v554: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-20T06:44:25.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:25 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4025273429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:25.718 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:25 smithi183 ceph-mon[22204]: pgmap v554: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 767 B/s wr, 9 op/s 2024-08-20T06:44:25.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:25 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4025273429' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 -- 172.21.15.107:0/151304648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 msgr2=0x7efcdc10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 --2- 172.21.15.107:0/151304648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc10f290 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7efcd4009ca0 tx=0x7efcd402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 -- 172.21.15.107:0/151304648 shutdown_connections 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 --2- 172.21.15.107:0/151304648 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc10f290 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 --2- 172.21.15.107:0/151304648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 0x7efcdc10c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 --2- 172.21.15.107:0/151304648 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 0x7efcdc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 -- 172.21.15.107:0/151304648 >> 172.21.15.107:0/151304648 conn(0x7efcdc0fd420 msgr2=0x7efcdc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 -- 172.21.15.107:0/151304648 shutdown_connections 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.157+0000 7efce4106640 1 -- 172.21.15.107:0/151304648 wait complete. 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 Processor -- start 2024-08-20T06:44:26.164 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 -- start start 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 0x7efcdc198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 0x7efcdc199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc1129a0 con 0x7efcdc10ce20 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc112820 con 0x7efcdc103870 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce4106640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7efcdc112b20 con 0x7efcdc104270 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39314/0 (socket says 172.21.15.107:39314) 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 -- 172.21.15.107:0/1076314932 learned_addr learned my addr 172.21.15.107:0/1076314932 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce1ea2640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 0x7efcdc198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce16a1640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 0x7efcdc199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:26.165 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 -- 172.21.15.107:0/1076314932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 msgr2=0x7efcdc199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 0x7efcdc199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 -- 172.21.15.107:0/1076314932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 msgr2=0x7efcdc198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 0x7efcdc198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.158+0000 7efce26a3640 1 -- 172.21.15.107:0/1076314932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcdc19eee0 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efce26a3640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efcd4005cc0 tx=0x7efcd4032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4032e00 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7efcd40044d0 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd4030b70 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcdc19f170 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.159+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcdc1a5890 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.160+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcdc103c50 con 0x7efcdc10ce20 2024-08-20T06:44:26.166 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.161+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7efcd4030d10 con 0x7efcdc10ce20 2024-08-20T06:44:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.162+0000 7efcd2ffd640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 0x7efcc80b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.162+0000 7efce1ea2640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 0x7efcc80b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.162+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7efcd41052c0 con 0x7efcdc10ce20 2024-08-20T06:44:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.163+0000 7efce1ea2640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 0x7efcc80b6090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcd80058c0 tx=0x7efcd800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:26.167 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.165+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efcd40cc910 con 0x7efcdc10ce20 2024-08-20T06:44:26.343 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.341+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efcdc1a5b30 con 0x7efcdc10ce20 2024-08-20T06:44:26.344 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.343+0000 7efcd2ffd640 1 -- 172.21.15.107:0/1076314932 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+22851 (secure 0 0 0) 0x7efcd40cc210 con 0x7efcdc10ce20 2024-08-20T06:44:26.344 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-08-20T06:44:26.345 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":73,"fsid":"1c223f02-5ebd-11ef-bccf-c7b262605968","created":"2024-08-20T06:27:17.087848+0000","modified":"2024-08-20T06:36:03.117597+0000","last_up_change":"2024-08-20T06:34:10.912656+0000","last_in_change":"2024-08-20T06:33:48.634366+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T06:30:39.511744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T06:34:56.266774+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T06:34:58.530080+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fb671ab-2dd3-4934-9bf5-4492165e868c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6803","nonce":3184473884}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6805","nonce":3184473884}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6809","nonce":3184473884}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":3184473884},{"type":"v1","addr":"172.21.15.107:6807","nonce":3184473884}]},"public_addr":"172.21.15.107:6803/3184473884","cluster_addr":"172.21.15.107:6805/3184473884","heartbeat_back_addr":"172.21.15.107:6809/3184473884","heartbeat_front_addr":"172.21.15.107:6807/3184473884","state":["exists","up"]},{"osd":1,"uuid":"7896e40d-68d0-4ca3-9a69-5364d3804903","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":12,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6801","nonce":2162362081}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6803","nonce":2162362081}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6807","nonce":2162362081}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2162362081},{"type":"v1","addr":"172.21.15.183:6805","nonce":2162362081}]},"public_addr":"172.21.15.183:6801/2162362081","cluster_addr":"172.21.15.183:6803/2162362081","heartbeat_back_addr":"172.21.15.183:6807/2162362081","heartbeat_front_addr":"172.21.15.183:6805/2162362081","state":["exists","up"]},{"osd":2,"uuid":"371c31f4-04e5-4bce-87c7-b8db82e15d1c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6801","nonce":4087892550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6803","nonce":4087892550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6807","nonce":4087892550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":4087892550},{"type":"v1","addr":"172.21.15.193:6805","nonce":4087892550}]},"public_addr":"172.21.15.193:6801/4087892550","cluster_addr":"172.21.15.193:6803/4087892550","heartbeat_back_addr":"172.21.15.193:6807/4087892550","heartbeat_front_addr":"172.21.15.193:6805/4087892550","state":["exists","up"]},{"osd":3,"uuid":"89b8f09d-16d2-4c69-8d1e-d2835a148199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":23,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6811","nonce":3836170124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6813","nonce":3836170124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6817","nonce":3836170124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":3836170124},{"type":"v1","addr":"172.21.15.107:6815","nonce":3836170124}]},"public_addr":"172.21.15.107:6811/3836170124","cluster_addr":"172.21.15.107:6813/3836170124","heartbeat_back_addr":"172.21.15.107:6817/3836170124","heartbeat_front_addr":"172.21.15.107:6815/3836170124","state":["exists","up"]},{"osd":4,"uuid":"1047c4ef-e210-49ca-86d1-84d0b9af15e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":29,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6809","nonce":3081917594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6811","nonce":3081917594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6815","nonce":3081917594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":3081917594},{"type":"v1","addr":"172.21.15.183:6813","nonce":3081917594}]},"public_addr":"172.21.15.183:6809/3081917594","cluster_addr":"172.21.15.183:6811/3081917594","heartbeat_back_addr":"172.21.15.183:6815/3081917594","heartbeat_front_addr":"172.21.15.183:6813/3081917594","state":["exists","up"]},{"osd":5,"uuid":"6e114967-5347-4fb6-a53b-db12e51cd2c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6809","nonce":2859012994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6811","nonce":2859012994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6815","nonce":2859012994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2859012994},{"type":"v1","addr":"172.21.15.193:6813","nonce":2859012994}]},"public_addr":"172.21.15.193:6809/2859012994","cluster_addr":"172.21.15.193:6811/2859012994","heartbeat_back_addr":"172.21.15.193:6815/2859012994","heartbeat_front_addr":"172.21.15.193:6813/2859012994","state":["exists","up"]},{"osd":6,"uuid":"56c0c67a-ffc9-49e3-976f-4a1c19cc6786","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6819","nonce":2830948691}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6821","nonce":2830948691}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6825","nonce":2830948691}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":2830948691},{"type":"v1","addr":"172.21.15.107:6823","nonce":2830948691}]},"public_addr":"172.21.15.107:6819/2830948691","cluster_addr":"172.21.15.107:6821/2830948691","heartbeat_back_addr":"172.21.15.107:6825/2830948691","heartbeat_front_addr":"172.21.15.107:6823/2830948691","state":["exists","up"]},{"osd":7,"uuid":"2ed828c5-26a6-488d-a45a-18937c0520cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":45,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6817","nonce":3897620215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6818","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6819","nonce":3897620215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6822","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6823","nonce":3897620215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":3897620215},{"type":"v1","addr":"172.21.15.183:6821","nonce":3897620215}]},"public_addr":"172.21.15.183:6817/3897620215","cluster_addr":"172.21.15.183:6819/3897620215","heartbeat_back_addr":"172.21.15.183:6823/3897620215","heartbeat_front_addr":"172.21.15.183:6821/3897620215","state":["exists","up"]},{"osd":8,"uuid":"1f41a3e6-90c8-46c8-bcd6-883e5b7a253a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6817","nonce":2908108197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6819","nonce":2908108197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6823","nonce":2908108197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":2908108197},{"type":"v1","addr":"172.21.15.193:6821","nonce":2908108197}]},"public_addr":"172.21.15.193:6817/2908108197","cluster_addr":"172.21.15.193:6819/2908108197","heartbeat_back_addr":"172.21.15.193:6823/2908108197","heartbeat_front_addr":"172.21.15.193:6821/2908108197","state":["exists","up"]},{"osd":9,"uuid":"ec582cc0-f65e-403c-bd4f-ab7077dd1f38","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6827","nonce":1423000516}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6829","nonce":1423000516}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6833","nonce":1423000516}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":1423000516},{"type":"v1","addr":"172.21.15.107:6831","nonce":1423000516}]},"public_addr":"172.21.15.107:6827/1423000516","cluster_addr":"172.21.15.107:6829/1423000516","heartbeat_back_addr":"172.21.15.107:6833/1423000516","heartbeat_front_addr":"172.21.15.107:6831/1423000516","state":["exists","up"]},{"osd":10,"uuid":"2a94743b-6ba0-4a13-878b-a5e931783b7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6825","nonce":2667647347}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6826","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6827","nonce":2667647347}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6830","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6831","nonce":2667647347}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6828","nonce":2667647347},{"type":"v1","addr":"172.21.15.183:6829","nonce":2667647347}]},"public_addr":"172.21.15.183:6825/2667647347","cluster_addr":"172.21.15.183:6827/2667647347","heartbeat_back_addr":"172.21.15.183:6831/2667647347","heartbeat_front_addr":"172.21.15.183:6829/2667647347","state":["exists","up"]},{"osd":11,"uuid":"e25cf09b-b923-401d-9d8b-16f2cfff57bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6825","nonce":735254759}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6827","nonce":735254759}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6831","nonce":735254759}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":735254759},{"type":"v1","addr":"172.21.15.193:6829","nonce":735254759}]},"public_addr":"172.21.15.193:6825/735254759","cluster_addr":"172.21.15.193:6827/735254759","heartbeat_back_addr":"172.21.15.193:6831/735254759","heartbeat_front_addr":"172.21.15.193:6829/735254759","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:29:50.332979+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:13.223120+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:30:36.245770+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:00.728777+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:25.025816+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:31:47.270490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:11.129754+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:34.710760+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:32:57.396479+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:23.013021+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:33:45.661862+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T06:34:09.649605+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"3.c","mappings":[{"from":3,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":6}]},{"pgid":"3.39","mappings":[{"from":5,"to":8}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1518535638":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/2483940460":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3373920322":"2024-08-21T06:28:01.208606+0000","172.21.15.107:6801/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/3013940704":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6801/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/235474616":"2024-08-21T06:27:38.122407+0000","172.21.15.107:0/484222332":"2024-08-21T06:27:38.122407+0000","172.21.15.107:6800/3759743791":"2024-08-21T06:28:01.208606+0000","172.21.15.107:0/343023724":"2024-08-21T06:27:38.122407+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T06:44:26.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 msgr2=0x7efcc80b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:26.346 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 0x7efcc80b6090 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efcd80058c0 tx=0x7efcd800a2d0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 msgr2=0x7efcdc19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7efcd4005cc0 tx=0x7efcd4032a10 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 shutdown_connections 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7efcdc10ce20 0x7efcdc19e710 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7efcc80b3bd0 0x7efcc80b6090 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcdc104270 0x7efcdc199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 --2- 172.21.15.107:0/1076314932 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7efcdc103870 0x7efcdc198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 >> 172.21.15.107:0/1076314932 conn(0x7efcdc0fd420 msgr2=0x7efcdc10f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 shutdown_connections 2024-08-20T06:44:26.347 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:26.345+0000 7efce4106640 1 -- 172.21.15.107:0/1076314932 wait complete. 2024-08-20T06:44:27.099 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-20T06:44:27.350 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:27 smithi107 ceph-mon[18495]: pgmap v555: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:44:27.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:27 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/1076314932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:44:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:27 smithi193 ceph-mon[21944]: pgmap v555: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:44:27.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:27 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/1076314932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:44:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:27 smithi183 ceph-mon[22204]: pgmap v555: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T06:44:27.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:27 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/1076314932' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T06:44:27.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 -- 172.21.15.107:0/2262454593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05301073a0 msgr2=0x7f05301077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:27.822 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 --2- 172.21.15.107:0/2262454593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05301073a0 0x7f05301077e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0524009900 tx=0x7f052402a410 comp rx=0 tx=0).stop 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 -- 172.21.15.107:0/2262454593 shutdown_connections 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 --2- 172.21.15.107:0/2262454593 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f0530107d20 0x7f053010e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 --2- 172.21.15.107:0/2262454593 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f05301073a0 0x7f05301077e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 --2- 172.21.15.107:0/2262454593 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 0x7f0530101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 -- 172.21.15.107:0/2262454593 >> 172.21.15.107:0/2262454593 conn(0x7f05300fd400 msgr2=0x7f05300ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 -- 172.21.15.107:0/2262454593 shutdown_connections 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.816+0000 7f0535f45640 1 -- 172.21.15.107:0/2262454593 wait complete. 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 Processor -- start 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 -- start start 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 0x7f0530197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05301073a0 0x7f0530198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:27.823 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f0530111890 con 0x7f0530107d20 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f0530111710 con 0x7f05301073a0 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.817+0000 7f0535f45640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f0530111a10 con 0x7f0530101990 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39328/0 (socket says 172.21.15.107:39328) 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 -- 172.21.15.107:0/920372422 learned_addr learned my addr 172.21.15.107:0/920372422 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052effd640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05301073a0 0x7f0530198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052f7fe640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 0x7f0530197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 -- 172.21.15.107:0/920372422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 msgr2=0x7f0530197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 0x7f0530197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 -- 172.21.15.107:0/920372422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05301073a0 msgr2=0x7f0530198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:27.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05301073a0 0x7f0530198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 -- 172.21.15.107:0/920372422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053019de70 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052ffff640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f052800cad0 tx=0x7f052800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052800ae20 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053019e160 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f0528005020 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052800f6b0 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.818+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05301a4820 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.819+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0530101d70 con 0x7f0530107d20 2024-08-20T06:44:27.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.821+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f05280051d0 con 0x7f0530107d20 2024-08-20T06:44:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.822+0000 7f052cff9640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 0x7f05180b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.822+0000 7f052f7fe640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 0x7f05180b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.822+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 8240+0+0 (secure 0 0 0) 0x7f05280e3f90 con 0x7f0530107d20 2024-08-20T06:44:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.823+0000 7f052f7fe640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 0x7f05180b6090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05200058c0 tx=0x7f052000a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:27.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:27.825+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0528014030 con 0x7f0530107d20 2024-08-20T06:44:28.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.039+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0530198ef0 con 0x7f0530107d20 2024-08-20T06:44:28.042 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.041+0000 7f052cff9640 1 -- 172.21.15.107:0/920372422 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) ==== 161+0+0 (secure 0 0 0) 0x7f0530198ef0 con 0x7f0530107d20 2024-08-20T06:44:28.042 INFO:teuthology.orchestra.run.smithi107.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 msgr2=0x7f05180b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 0x7f05180b6090 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05200058c0 tx=0x7f052000a4c0 comp rx=0 tx=0).stop 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 msgr2=0x7f053019d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f052800cad0 tx=0x7f052800cf90 comp rx=0 tx=0).stop 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 shutdown_connections 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0530107d20 0x7f053019d6a0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f05301073a0 0x7f0530198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.044+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f05180b3bd0 0x7f05180b6090 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:28.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.045+0000 7f0535f45640 1 --2- 172.21.15.107:0/920372422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0530101990 0x7f0530197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:28.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.045+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 >> 172.21.15.107:0/920372422 conn(0x7f05300fd400 msgr2=0x7f05300ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:28.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.045+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 shutdown_connections 2024-08-20T06:44:28.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:28.045+0000 7f0535f45640 1 -- 172.21.15.107:0/920372422 wait complete. 2024-08-20T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a[18491]: 2024-08-20T06:44:28.040+0000 7f3a7dcdf640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T06:44:28.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T06:44:28.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T06:44:28.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T06:44:28.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:28 smithi107 ceph-mon[18495]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T06:44:28.486 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.486 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.486 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mds[51641]: -- 172.21.15.183:0/1964419856 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1d24e9860 con 0x55d1d24c7400 2024-08-20T06:44:28.486 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: starting mds.h at 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mon[22204]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.487 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mds[51957]: -- 172.21.15.183:0/827013401 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55df9b22d860 con 0x55df9b20b400 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: starting mds.b at 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-mds[51336]: -- 172.21.15.183:0/1062905746 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac04bd5860 con 0x55ac04bb3400 2024-08-20T06:44:28.488 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:44:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: starting mds.e at 2024-08-20T06:44:28.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mds[52861]: -- 172.21.15.193:0/1100875809 <== mon.2 v2:172.21.15.183:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bae935bc20 con 0x55bae9339000 2024-08-20T06:44:28.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: starting mds.i at 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mds[52561]: -- 172.21.15.193:0/2349026103 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b625845860 con 0x55b625823000 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: starting mds.c at 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T06:44:28.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/920372422' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mon[21944]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: ignoring --setuser ceph since I am not root 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: ignoring --setgroup ceph since I am not root 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-mds[52259]: -- 172.21.15.193:0/2759086053 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2ddf3b860 con 0x55c2ddf18800 2024-08-20T06:44:28.556 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:44:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: starting mds.f at 2024-08-20T06:44:28.731 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-20T06:44:28.975 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: pgmap v556: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6834/3571582191,v1:172.21.15.183:6835/3571582191] up:boot 2024-08-20T06:44:29.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6834/2138990247,v1:172.21.15.193:6835/2138990247] up:boot 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6836/2486939634,v1:172.21.15.183:6837/2486939634] up:boot 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6836/3865706021,v1:172.21.15.193:6837/3865706021] up:boot 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.193:6838/1768203246,v1:172.21.15.193:6839/1768203246] up:boot 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: mds.? [v2:172.21.15.183:6838/3219821074,v1:172.21.15.183:6839/3219821074] up:boot 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:44:29.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:29 smithi107 ceph-mon[18495]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:44:29.530 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 -- 172.21.15.107:0/2184655269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824102aa0 msgr2=0x7f8824102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 --2- 172.21.15.107:0/2184655269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824102aa0 0x7f8824102e80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f8814009ca0 tx=0x7f881402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 -- 172.21.15.107:0/2184655269 shutdown_connections 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 --2- 172.21.15.107:0/2184655269 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f8824108e30 0x7f882410f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 --2- 172.21.15.107:0/2184655269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f88241084b0 0x7f88241088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 --2- 172.21.15.107:0/2184655269 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824102aa0 0x7f8824102e80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.419+0000 7f8829930640 1 -- 172.21.15.107:0/2184655269 >> 172.21.15.107:0/2184655269 conn(0x7f88240fe550 msgr2=0x7f8824100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 -- 172.21.15.107:0/2184655269 shutdown_connections 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 -- 172.21.15.107:0/2184655269 wait complete. 2024-08-20T06:44:29.531 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 Processor -- start 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 -- start start 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8824102aa0 0x7f882419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.420+0000 7f8829930640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 0x7f88241a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8829930640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f88241129f0 con 0x7f8824108e30 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8829930640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f8824112870 con 0x7f88241084b0 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8829930640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f8824112b70 con 0x7f8824102aa0 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.183:3300/0 says I am v2:172.21.15.107:42314/0 (socket says 172.21.15.107:42314) 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 -- 172.21.15.107:0/4046267333 learned_addr learned my addr 172.21.15.107:0/4046267333 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88237fe640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 0x7f88241a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8822ffd640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8824102aa0 0x7f882419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 -- 172.21.15.107:0/4046267333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8824102aa0 msgr2=0x7f882419b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:29.532 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8824102aa0 0x7f882419b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 -- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 msgr2=0x7f88241a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 0x7f88241a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 -- 172.21.15.107:0/4046267333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88241a1190 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88237fe640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 0x7f88241a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f88227fc640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e9f0 tx=0x7f880c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c00cd80 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f880c0054b0 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c010430 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88241a1480 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.421+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a7b40 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.422+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8824102e80 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.423+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f880c0046c0 con 0x7f88241084b0 2024-08-20T06:44:29.533 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.424+0000 7f882892e640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 0x7f88100b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:29.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.424+0000 7f8822ffd640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 0x7f88100b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:29.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.424+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8498+0+0 (secure 0 0 0) 0x7f880c0e40d0 con 0x7f88241084b0 2024-08-20T06:44:29.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.425+0000 7f8822ffd640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 0x7f88100b6090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8814000f80 tx=0x7f8814031000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:29.534 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.429+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f880c0ab5b0 con 0x7f88241084b0 2024-08-20T06:44:29.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: pgmap v556: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:29.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6834/3571582191,v1:172.21.15.183:6835/3571582191] up:boot 2024-08-20T06:44:29.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6834/2138990247,v1:172.21.15.193:6835/2138990247] up:boot 2024-08-20T06:44:29.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6836/2486939634,v1:172.21.15.183:6837/2486939634] up:boot 2024-08-20T06:44:29.599 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6836/3865706021,v1:172.21.15.193:6837/3865706021] up:boot 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.193:6838/1768203246,v1:172.21.15.193:6839/1768203246] up:boot 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: mds.? [v2:172.21.15.183:6838/3219821074,v1:172.21.15.183:6839/3219821074] up:boot 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:44:29.600 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:29 smithi193 ceph-mon[21944]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:44:29.646 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:29.643+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f882410ea90 con 0x7f88241084b0 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: pgmap v556: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 767 B/s wr, 7 op/s 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6834/3571582191,v1:172.21.15.183:6835/3571582191] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6834/2138990247,v1:172.21.15.193:6835/2138990247] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6836/2486939634,v1:172.21.15.183:6837/2486939634] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6836/3865706021,v1:172.21.15.193:6837/3865706021] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.193:6838/1768203246,v1:172.21.15.193:6839/1768203246] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: mds.? [v2:172.21.15.183:6838/3219821074,v1:172.21.15.183:6839/3219821074] up:boot 2024-08-20T06:44:29.737 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T06:44:29.738 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:29 smithi183 ceph-mon[22204]: from='mgr.14150 172.21.15.107:0/1076923688' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T06:44:30.293 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.291+0000 7f882892e640 1 -- 172.21.15.107:0/4046267333 <== mon.2 v2:172.21.15.183:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f880c0aaeb0 con 0x7f88241084b0 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 msgr2=0x7f88100b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 0x7f88100b6090 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8814000f80 tx=0x7f8814031000 comp rx=0 tx=0).stop 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 msgr2=0x7f882419b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f880c00e9f0 tx=0x7f880c00edf0 comp rx=0 tx=0).stop 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 shutdown_connections 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8824108e30 0x7f88241a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:30.296 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f88241084b0 0x7f882419b620 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:30.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f88100b3bd0 0x7f88100b6090 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:30.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 --2- 172.21.15.107:0/4046267333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8824102aa0 0x7f882419b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:30.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 >> 172.21.15.107:0/4046267333 conn(0x7f88240fe550 msgr2=0x7f8824104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:30.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 shutdown_connections 2024-08-20T06:44:30.297 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:30.295+0000 7f8829930640 1 -- 172.21.15.107:0/4046267333 wait complete. 2024-08-20T06:44:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:30 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/4046267333' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:30 smithi193 ceph-mon[21944]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:30 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/4046267333' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:30 smithi107 ceph-mon[18495]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:30 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/4046267333' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:30 smithi183 ceph-mon[22204]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T06:44:30.919 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-20T06:44:31.170 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: pgmap v558: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: Cluster is now healthy 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:44:31.539 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:31 smithi107 ceph-mon[18495]: fsmap 12 up:standby 2024-08-20T06:44:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: pgmap v558: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-20T06:44:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T06:44:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T06:44:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: Cluster is now healthy 2024-08-20T06:44:31.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:44:31.555 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:31 smithi193 ceph-mon[21944]: fsmap 12 up:standby 2024-08-20T06:44:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.621+0000 7f2fae979640 1 -- 172.21.15.107:0/2365307300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 msgr2=0x7f2fa8103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.621+0000 7f2fae979640 1 --2- 172.21.15.107:0/2365307300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8103d80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009900 tx=0x7f2f9802a410 comp rx=0 tx=0).stop 2024-08-20T06:44:31.639 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 -- 172.21.15.107:0/2365307300 shutdown_connections 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 --2- 172.21.15.107:0/2365307300 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa810ce00 0x7f2fa810f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 --2- 172.21.15.107:0/2365307300 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa81042c0 0x7f2fa810c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 --2- 172.21.15.107:0/2365307300 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8103d80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 -- 172.21.15.107:0/2365307300 >> 172.21.15.107:0/2365307300 conn(0x7f2fa80fd400 msgr2=0x7f2fa80ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 -- 172.21.15.107:0/2365307300 shutdown_connections 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 -- 172.21.15.107:0/2365307300 wait complete. 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 Processor -- start 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 -- start start 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.622+0000 7f2fae979640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa81042c0 0x7f2fa81993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 0x7f2fa819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8112960 con 0x7f2fa81039a0 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa81127e0 con 0x7f2fa81042c0 2024-08-20T06:44:31.640 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa8112ae0 con 0x7f2fa810ce00 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39358/0 (socket says 172.21.15.107:39358) 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 -- 172.21.15.107:0/2089785451 learned_addr learned my addr 172.21.15.107:0/2089785451 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa77fe640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa81042c0 0x7f2fa81993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 -- 172.21.15.107:0/2089785451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 msgr2=0x7f2fa819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2facf16640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 0x7f2fa819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 0x7f2fa819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 -- 172.21.15.107:0/2089785451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa81042c0 msgr2=0x7f2fa81993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa81042c0 0x7f2fa81993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 -- 172.21.15.107:0/2089785451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa819ef40 con 0x7f2fa81039a0 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2facf16640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 0x7f2fa819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa7fff640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009a20 tx=0x7f2f98004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:31.641 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9803f030 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7f2f980035f0 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f98035070 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa819f1d0 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.623+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa81a58f0 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.624+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa8103d80 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.626+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7f2f98003aa0 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.626+0000 7f2fa57fa640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 0x7f2f900b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.626+0000 7f2fa77fe640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 0x7f2f900b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.626+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8498+0+0 (secure 0 0 0) 0x7f2f98104c40 con 0x7f2fa81039a0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.627+0000 7f2fa77fe640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 0x7f2f900b6090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80ffc10 tx=0x7f2f9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:31.642 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.629+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2f980cc180 con 0x7f2fa81039a0 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: pgmap v558: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: Cluster is now healthy 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T06:44:31.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:31 smithi183 ceph-mon[22204]: fsmap 12 up:standby 2024-08-20T06:44:31.807 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:31.805+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2fa81a5bf0 con 0x7f2fa81039a0 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.294+0000 7f2fa57fa640 1 -- 172.21.15.107:0/2089785451 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7f2f980cba80 con 0x7f2fa81039a0 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_metadata' removed 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 msgr2=0x7f2f900b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 0x7f2f900b6090 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80ffc10 tx=0x7f2f9c005f90 comp rx=0 tx=0).stop 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 msgr2=0x7f2fa8198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:32.298 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009a20 tx=0x7f2f98004270 comp rx=0 tx=0).stop 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 shutdown_connections 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7f2f900b3bd0 0x7f2f900b6090 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2fa810ce00 0x7f2fa819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7f2fa81042c0 0x7f2fa81993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 --2- 172.21.15.107:0/2089785451 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2fa81039a0 0x7f2fa8198e90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 >> 172.21.15.107:0/2089785451 conn(0x7f2fa80fd400 msgr2=0x7f2fa810f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 shutdown_connections 2024-08-20T06:44:32.299 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:32.297+0000 7f2fae979640 1 -- 172.21.15.107:0/2089785451 wait complete. 2024-08-20T06:44:32.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:32 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:32.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:32 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:32.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:32 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:33.080 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-20T06:44:33.324 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T06:44:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:33 smithi107 ceph-mon[18495]: pgmap v559: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:44:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:33 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:33.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:33 smithi107 ceph-mon[18495]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T06:44:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:33 smithi193 ceph-mon[21944]: pgmap v559: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:44:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:33 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:33.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:33 smithi193 ceph-mon[21944]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T06:44:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:33 smithi183 ceph-mon[22204]: pgmap v559: 97 pgs: 97 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-08-20T06:44:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:33 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/2089785451' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:33.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:33 smithi183 ceph-mon[22204]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 -- 172.21.15.107:0/3877136462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c10f0c0 msgr2=0x7ff56c1114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 --2- 172.21.15.107:0/3877136462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1114b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff564009ca0 tx=0x7ff56402a8c0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 -- 172.21.15.107:0/3877136462 shutdown_connections 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 --2- 172.21.15.107:0/3877136462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1114b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 --2- 172.21.15.107:0/3877136462 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c075b30 0x7ff56c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 --2- 172.21.15.107:0/3877136462 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 0x7ff56c0755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 -- 172.21.15.107:0/3877136462 >> 172.21.15.107:0/3877136462 conn(0x7ff56c0fe540 msgr2=0x7ff56c100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:33.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 -- 172.21.15.107:0/3877136462 shutdown_connections 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 -- 172.21.15.107:0/3877136462 wait complete. 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.798+0000 7ff570f98640 1 Processor -- start 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 -- start start 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 --2- >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 0x7ff56c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c114be0 con 0x7ff56c075b30 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 -- --> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c114a60 con 0x7ff56c075210 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff570f98640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c114d60 con 0x7ff56c10f0c0 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:39364/0 (socket says 172.21.15.107:39364) 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 -- 172.21.15.107:0/3213402912 learned_addr learned my addr 172.21.15.107:0/3213402912 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-08-20T06:44:33.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff56a59c640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 0x7ff56c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff56ad9d640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 -- 172.21.15.107:0/3213402912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c10f0c0 msgr2=0x7ff56c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 -- 172.21.15.107:0/3213402912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 msgr2=0x7ff56c19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 0x7ff56c19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 -- 172.21.15.107:0/3213402912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c1a1240 con 0x7ff56c075b30 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.799+0000 7ff561d9b640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff55400b4a0 tx=0x7ff55400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.800+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554014070 con 0x7ff56c075b30 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.800+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56c1a1530 con 0x7ff56c075b30 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.800+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(25 keys) ==== 1063+0+0 (secure 0 0 0) 0x7ff554010aa0 con 0x7ff56c075b30 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.800+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff554017b70 con 0x7ff56c075b30 2024-08-20T06:44:33.818 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.800+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56c1a7bf0 con 0x7ff56c075b30 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.801+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56c0755f0 con 0x7ff56c075b30 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.802+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 20) ==== 151060+0+0 (secure 0 0 0) 0x7ff554017070 con 0x7ff56c075b30 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.803+0000 7ff563fff640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 0x7ff5580b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.803+0000 7ff56a59c640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 0x7ff5580b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.803+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7944+0+0 (secure 0 0 0) 0x7ff5540e3220 con 0x7ff56c075b30 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.803+0000 7ff56a59c640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 0x7ff5580b6090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00bbe0 tx=0x7ff55c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T06:44:33.819 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.806+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff5540aa9e0 con 0x7ff56c075b30 2024-08-20T06:44:33.983 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:33.982+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff56c1a7f50 con 0x7ff56c075b30 2024-08-20T06:44:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.302+0000 7ff563fff640 1 -- 172.21.15.107:0/3213402912 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7ff5540aa2e0 con 0x7ff56c075b30 2024-08-20T06:44:34.303 INFO:teuthology.orchestra.run.smithi107.stderr:pool 'cephfs_data' removed 2024-08-20T06:44:34.305 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 msgr2=0x7ff5580b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 0x7ff5580b6090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff55c00bbe0 tx=0x7ff55c00b390 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 msgr2=0x7ff56c19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff55400b4a0 tx=0x7ff55400b960 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 shutdown_connections 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:6800/1083091367,v1:172.21.15.107:6801/1083091367] conn(0x7ff5580b3bd0 0x7ff5580b6090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff56c10f0c0 0x7ff56c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff56c075b30 0x7ff56c19b6d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 --2- 172.21.15.107:0/3213402912 >> [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] conn(0x7ff56c075210 0x7ff56c19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.304+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 >> 172.21.15.107:0/3213402912 conn(0x7ff56c0fe540 msgr2=0x7ff56c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.305+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 shutdown_connections 2024-08-20T06:44:34.306 INFO:teuthology.orchestra.run.smithi107.stderr:2024-08-20T06:44:34.305+0000 7ff570f98640 1 -- 172.21.15.107:0/3213402912 wait complete. 2024-08-20T06:44:34.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:34 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:34.611 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:34 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:34.736 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:34 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T06:44:35.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:44:35.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:44:35.170 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:44:35.206 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-20T06:44:35.206 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T06:44:35.222 DEBUG:teuthology.orchestra.run.smithi183:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T06:44:35.239 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T06:44:35.264 INFO:tasks.cephadm:Stopping all daemons... 2024-08-20T06:44:35.264 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-20T06:44:35.264 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a 2024-08-20T06:44:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 ceph-mon[18495]: pgmap v561: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:44:35.361 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 ceph-mon[18495]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:35.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 ceph-mon[18495]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T06:44:35.362 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 systemd[1]: Stopping Ceph mon.a for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:35 smithi183 ceph-mon[22204]: pgmap v561: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:44:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:35 smithi183 ceph-mon[22204]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:35.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:35 smithi183 ceph-mon[22204]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T06:44:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:35 smithi193 ceph-mon[21944]: pgmap v561: 65 pgs: 65 active+clean; 577 KiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T06:44:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:35 smithi193 ceph-mon[21944]: from='client.? 172.21.15.107:0/3213402912' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T06:44:35.554 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:35 smithi193 ceph-mon[21944]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T06:44:35.858 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a[18491]: 2024-08-20T06:44:35.548+0000 7f3a834ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:35.859 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a[18491]: 2024-08-20T06:44:35.548+0000 7f3a834ea640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-20T06:44:35.859 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 podman[331416]: 2024-08-20 06:44:35.580679614 +0000 UTC m=+0.128416681 container died b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-08-20T06:44:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 podman[331416]: 2024-08-20 06:44:35.949107797 +0000 UTC m=+0.496844854 container cleanup b8c843f327a7242ebe2a28241f93d8a9c31c1e59751810b758b80c050082190d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-20T06:44:36.111 INFO:journalctl@ceph.mon.a.smithi107.stdout:Aug 20 06:44:35 smithi107 bash[331416]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-a 2024-08-20T06:44:36.522 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.a.service' 2024-08-20T06:44:36.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:36.574 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-20T06:44:36.574 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-20T06:44:36.574 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.b 2024-08-20T06:44:36.875 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:36 smithi183 systemd[1]: Stopping Ceph mon.b for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:37.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b[22200]: 2024-08-20T06:44:36.873+0000 7f89d59e4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:37.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b[22200]: 2024-08-20T06:44:36.873+0000 7f89d59e4640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-20T06:44:37.217 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:36 smithi183 podman[55419]: 2024-08-20 06:44:36.935757061 +0000 UTC m=+0.167993669 container died a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-08-20T06:44:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:37 smithi183 podman[55419]: 2024-08-20 06:44:37.299352112 +0000 UTC m=+0.531588717 container cleanup a129e4ad13be07b31fe7df5364f236f34150749f10ff0c30fb01bf55a30bfa56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:44:37.486 INFO:journalctl@ceph.mon.b.smithi183.stdout:Aug 20 06:44:37 smithi183 bash[55419]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-b 2024-08-20T06:44:37.733 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.b.service' 2024-08-20T06:44:37.790 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:37.790 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-20T06:44:37.790 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-20T06:44:37.790 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.c 2024-08-20T06:44:38.093 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:37 smithi193 systemd[1]: Stopping Ceph mon.c for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:38.535 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c[21940]: 2024-08-20T06:44:38.091+0000 7f578f7f7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:38.535 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c[21940]: 2024-08-20T06:44:38.091+0000 7f578f7f7640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-20T06:44:38.535 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:38 smithi193 podman[56717]: 2024-08-20 06:44:38.276472151 +0000 UTC m=+0.285438432 container died c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-20T06:44:38.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:38 smithi193 podman[56717]: 2024-08-20 06:44:38.625935227 +0000 UTC m=+0.634901501 container cleanup c44fe035ac454053b4911bb0185b0749d1981bcc033bb0e81bae48b2c4592cac (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-08-20T06:44:38.804 INFO:journalctl@ceph.mon.c.smithi193.stdout:Aug 20 06:44:38 smithi193 bash[56717]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mon-c 2024-08-20T06:44:39.048 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mon.c.service' 2024-08-20T06:44:39.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:39.092 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-20T06:44:39.093 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-20T06:44:39.093 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x 2024-08-20T06:44:39.413 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:44:39 smithi107 systemd[1]: Stopping Ceph mgr.x for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:39.713 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:44:39 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x[18724]: 2024-08-20T06:44:39.412+0000 7f8523475640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:39.713 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:44:39 smithi107 podman[331536]: 2024-08-20 06:44:39.443707238 +0000 UTC m=+0.155275711 container died b70a6c5efcff28242c8a573086e3a87bddb6095c3aa1c5ff7a0344b4315a3eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T06:44:40.090 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:44:39 smithi107 podman[331536]: 2024-08-20 06:44:39.81312689 +0000 UTC m=+0.524695362 container cleanup b70a6c5efcff28242c8a573086e3a87bddb6095c3aa1c5ff7a0344b4315a3eb5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:44:40.091 INFO:journalctl@ceph.mgr.x.smithi107.stdout:Aug 20 06:44:39 smithi107 bash[331536]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-x 2024-08-20T06:44:40.241 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-20T06:44:40.277 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:40.277 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-20T06:44:40.278 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-20T06:44:40.278 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.y 2024-08-20T06:44:40.609 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Aug 20 06:44:40 smithi183 systemd[1]: Stopping Ceph mgr.y for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:40.921 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Aug 20 06:44:40 smithi183 podman[55543]: 2024-08-20 06:44:40.608100269 +0000 UTC m=+0.125946467 container died c1775697a9ea195c6f7597ef37b12386d0a8c0efbe15d1bb4cd3200c486a5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:44:41.229 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Aug 20 06:44:40 smithi183 podman[55543]: 2024-08-20 06:44:40.995153665 +0000 UTC m=+0.512999864 container cleanup c1775697a9ea195c6f7597ef37b12386d0a8c0efbe15d1bb4cd3200c486a5e16 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:44:41.230 INFO:journalctl@ceph.mgr.y.smithi183.stdout:Aug 20 06:44:40 smithi183 bash[55543]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-y 2024-08-20T06:44:41.359 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-20T06:44:41.394 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:41.394 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-20T06:44:41.395 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-20T06:44:41.395 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.z 2024-08-20T06:44:41.744 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:41 smithi193 systemd[1]: Stopping Ceph mgr.z for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:42.054 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:41 smithi193 podman[56843]: 2024-08-20 06:44:41.743287357 +0000 UTC m=+0.151964519 container died b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T06:44:42.467 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:42 smithi193 podman[56843]: 2024-08-20 06:44:42.063791215 +0000 UTC m=+0.472468365 container cleanup b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-08-20T06:44:42.467 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:42 smithi193 bash[56843]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z 2024-08-20T06:44:42.467 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:42 smithi193 podman[56855]: 2024-08-20 06:44:42.265432166 +0000 UTC m=+0.516387445 container remove b4d8ddeaf9e15fc3b825245f0a9159c29d7be744089a975e165fc680dfa6d441 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-20T06:44:42.467 INFO:journalctl@ceph.mgr.z.smithi193.stdout:Aug 20 06:44:42 smithi193 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-08-20T06:44:42.486 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-20T06:44:42.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:42.521 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-20T06:44:42.521 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-20T06:44:42.521 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.0 2024-08-20T06:44:42.843 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:42 smithi107 systemd[1]: Stopping Ceph osd.0 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:43.111 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:44:42.842+0000 7f883c509640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:43.111 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:44:42.842+0000 7f883c509640 -1 osd.0 76 *** Got signal Terminated *** 2024-08-20T06:44:43.111 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0[32406]: 2024-08-20T06:44:42.842+0000 7f883c509640 -1 osd.0 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:44:48.569 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:48 smithi107 podman[331658]: 2024-08-20 06:44:48.285977774 +0000 UTC m=+5.574350859 container died 36853bc885a13e1955f34abddfa40b915e31c9e8ea8f3fff215c9c338b0f2ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-20T06:44:48.861 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:48 smithi107 podman[331658]: 2024-08-20 06:44:48.767951821 +0000 UTC m=+6.056324933 container cleanup 36853bc885a13e1955f34abddfa40b915e31c9e8ea8f3fff215c9c338b0f2ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:44:48.861 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:48 smithi107 bash[331658]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0 2024-08-20T06:44:49.361 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331675]: 2024-08-20 06:44:49.069551337 +0000 UTC m=+0.777859800 container remove 36853bc885a13e1955f34abddfa40b915e31c9e8ea8f3fff215c9c338b0f2ce8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-20T06:44:49.723 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.3326426 +0000 UTC m=+0.015677198 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:44:49.724 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.478784874 +0000 UTC m=+0.161819461 container create a1b7c96425698ac6af2413b3ef6d60cd82b3b697f589e7185e505b5d96680a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0-deactivate, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T06:44:50.112 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.723320093 +0000 UTC m=+0.406354683 container init a1b7c96425698ac6af2413b3ef6d60cd82b3b697f589e7185e505b5d96680a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:44:50.112 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.72763302 +0000 UTC m=+0.410667606 container start a1b7c96425698ac6af2413b3ef6d60cd82b3b697f589e7185e505b5d96680a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-20T06:44:50.112 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.837864134 +0000 UTC m=+0.520898758 container attach a1b7c96425698ac6af2413b3ef6d60cd82b3b697f589e7185e505b5d96680a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731) 2024-08-20T06:44:50.112 INFO:journalctl@ceph.osd.0.smithi107.stdout:Aug 20 06:44:49 smithi107 podman[331764]: 2024-08-20 06:44:49.915826102 +0000 UTC m=+0.598860700 container died a1b7c96425698ac6af2413b3ef6d60cd82b3b697f589e7185e505b5d96680a2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-20T06:44:50.672 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.0.service' 2024-08-20T06:44:50.714 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:50.714 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-20T06:44:50.714 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-20T06:44:50.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.1 2024-08-20T06:44:51.016 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:50 smithi183 systemd[1]: Stopping Ceph osd.1 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:51.486 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:44:51.015+0000 7fd51429f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:51.486 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:44:51.015+0000 7fd51429f640 -1 osd.1 76 *** Got signal Terminated *** 2024-08-20T06:44:51.486 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1[26648]: 2024-08-20T06:44:51.015+0000 7fd51429f640 -1 osd.1 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:44:56.692 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:56 smithi183 podman[55682]: 2024-08-20 06:44:56.383804244 +0000 UTC m=+5.474964843 container died df9449258bae72bcccabba115a7f8dd77a570b8a8d5427aa9cd9835691519101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True) 2024-08-20T06:44:56.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:56 smithi183 podman[55682]: 2024-08-20 06:44:56.781948351 +0000 UTC m=+5.873108950 container cleanup df9449258bae72bcccabba115a7f8dd77a570b8a8d5427aa9cd9835691519101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-20T06:44:56.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:56 smithi183 bash[55682]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1 2024-08-20T06:44:57.316 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:56 smithi183 podman[55698]: 2024-08-20 06:44:56.991490547 +0000 UTC m=+0.603076261 container remove df9449258bae72bcccabba115a7f8dd77a570b8a8d5427aa9cd9835691519101 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:44:57.595 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.218472123 +0000 UTC m=+0.015511650 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:44:57.595 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.325657166 +0000 UTC m=+0.122696680 container create 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T06:44:57.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.594850626 +0000 UTC m=+0.391890141 container init 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T06:44:57.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.598464526 +0000 UTC m=+0.395504049 container start 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T06:44:57.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.686628826 +0000 UTC m=+0.483668353 container attach 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:44:57.986 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:57 smithi183 podman[55788]: 2024-08-20 06:44:57.784983777 +0000 UTC m=+0.582023303 container died 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T06:44:58.324 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:58 smithi183 podman[55811]: 2024-08-20 06:44:58.298339103 +0000 UTC m=+0.507590210 container remove 753be11c5c66e5a8f1de781f469cb6ebdeea4a2d21c741eeaa17a925457a591e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-1-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True) 2024-08-20T06:44:58.324 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:58 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.1.service: Deactivated successfully. 2024-08-20T06:44:58.324 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:58 smithi183 systemd[1]: Stopped Ceph osd.1 for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:44:58.324 INFO:journalctl@ceph.osd.1.smithi183.stdout:Aug 20 06:44:58 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.1.service: Consumed 30.069s CPU time. 2024-08-20T06:44:58.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.1.service' 2024-08-20T06:44:58.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:44:58.365 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-20T06:44:58.365 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-20T06:44:58.365 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.2 2024-08-20T06:44:58.694 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:44:58 smithi193 systemd[1]: Stopping Ceph osd.2 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:44:59.054 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:44:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:44:58.693+0000 7f205c54d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:44:59.054 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:44:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:44:58.693+0000 7f205c54d640 -1 osd.2 76 *** Got signal Terminated *** 2024-08-20T06:44:59.055 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:44:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2[26877]: 2024-08-20T06:44:58.693+0000 7f205c54d640 -1 osd.2 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:04.460 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:04 smithi193 podman[56980]: 2024-08-20 06:45:04.202499593 +0000 UTC m=+5.639449265 container died e610b98d865a2c54b21d11c67509bf61443e8958942f14f42c63085eaac2b22a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-20T06:45:04.719 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:04 smithi193 podman[56980]: 2024-08-20 06:45:04.534282862 +0000 UTC m=+5.971232534 container cleanup e610b98d865a2c54b21d11c67509bf61443e8958942f14f42c63085eaac2b22a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T06:45:04.719 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:04 smithi193 bash[56980]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2 2024-08-20T06:45:05.054 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:04 smithi193 podman[56998]: 2024-08-20 06:45:04.718816528 +0000 UTC m=+0.505619087 container remove e610b98d865a2c54b21d11c67509bf61443e8958942f14f42c63085eaac2b22a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True) 2024-08-20T06:45:05.405 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.15307751 +0000 UTC m=+0.111425052 container create 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-08-20T06:45:05.405 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.057954911 +0000 UTC m=+0.016302464 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:05.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.405609081 +0000 UTC m=+0.363956625 container init 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-08-20T06:45:05.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.409067 +0000 UTC m=+0.367414552 container start 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-08-20T06:45:05.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.488905532 +0000 UTC m=+0.447253094 container attach 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T06:45:05.805 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:05 smithi193 podman[57086]: 2024-08-20 06:45:05.597236749 +0000 UTC m=+0.555584301 container died 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-08-20T06:45:06.134 INFO:journalctl@ceph.osd.2.smithi193.stdout:Aug 20 06:45:06 smithi193 podman[57109]: 2024-08-20 06:45:06.117518363 +0000 UTC m=+0.511349337 container remove 6cc32186b31fa5f9881ac663a41d1ff67fbd57c851cc3eeae4b23b55cb02d990 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:45:06.140 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.2.service' 2024-08-20T06:45:06.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:06.180 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-20T06:45:06.180 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-20T06:45:06.180 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.3 2024-08-20T06:45:06.480 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:06 smithi107 systemd[1]: Stopping Ceph osd.3 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:06.861 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:06 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:45:06.478+0000 7ff9b97d9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:06.861 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:06 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:45:06.478+0000 7ff9b97d9640 -1 osd.3 76 *** Got signal Terminated *** 2024-08-20T06:45:06.861 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:06 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:45:06.478+0000 7ff9b97d9640 -1 osd.3 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:09.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:09 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:09.310+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:09.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:09.347+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:10.361 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:10 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:10.262+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:10.361 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:10 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:45:10.103+0000 7ff9b5dee640 -1 osd.3 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.773008+0000 front 2024-08-20T06:44:45.773054+0000 (oldest deadline 2024-08-20T06:45:09.873108+0000) 2024-08-20T06:45:10.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:10.384+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:10.805 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:10.457+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:11.543 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:11 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3[39306]: 2024-08-20T06:45:11.126+0000 7ff9b5dee640 -1 osd.3 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.773008+0000 front 2024-08-20T06:44:45.773054+0000 (oldest deadline 2024-08-20T06:45:09.873108+0000) 2024-08-20T06:45:11.543 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:11 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:11.299+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:11.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:11.467+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:11.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:11.336+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:11.814 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:11 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:11.542+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:12.096 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:11 smithi107 podman[331881]: 2024-08-20 06:45:11.814504773 +0000 UTC m=+5.439384207 container died 18c8925da28e3f3fe1ff4cc13013b1750def6b9fb1e19163948150b6f13bd5bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-20T06:45:12.362 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:12 smithi107 podman[331881]: 2024-08-20 06:45:12.203958072 +0000 UTC m=+5.828837499 container cleanup 18c8925da28e3f3fe1ff4cc13013b1750def6b9fb1e19163948150b6f13bd5bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-20T06:45:12.362 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:12 smithi107 bash[331881]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3 2024-08-20T06:45:12.362 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:12 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:12.288+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:12.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:12.424+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:12.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:12.348+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:12.806 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:12 smithi107 podman[331900]: 2024-08-20 06:45:12.455266092 +0000 UTC m=+0.636269034 container remove 18c8925da28e3f3fe1ff4cc13013b1750def6b9fb1e19163948150b6f13bd5bb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-20T06:45:12.806 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:12 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:12.575+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:13.111 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:12 smithi107 podman[331987]: 2024-08-20 06:45:12.806069319 +0000 UTC m=+0.111081210 container create 10a77da9e004cbefc3d1bb624551d437e24d1294e06fb55594d9de2aed0c46b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T06:45:13.111 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:12 smithi107 podman[331987]: 2024-08-20 06:45:12.710783639 +0000 UTC m=+0.015795542 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:13.111 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:13 smithi107 podman[331987]: 2024-08-20 06:45:13.033569864 +0000 UTC m=+0.338581762 container init 10a77da9e004cbefc3d1bb624551d437e24d1294e06fb55594d9de2aed0c46b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:13.111 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:13 smithi107 podman[331987]: 2024-08-20 06:45:13.036923329 +0000 UTC m=+0.341935218 container start 10a77da9e004cbefc3d1bb624551d437e24d1294e06fb55594d9de2aed0c46b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:45:13.570 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:13.320+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:13.570 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:13 smithi107 podman[331987]: 2024-08-20 06:45:13.11677982 +0000 UTC m=+0.421791726 container attach 10a77da9e004cbefc3d1bb624551d437e24d1294e06fb55594d9de2aed0c46b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:13.570 INFO:journalctl@ceph.osd.3.smithi107.stdout:Aug 20 06:45:13 smithi107 podman[331987]: 2024-08-20 06:45:13.225272601 +0000 UTC m=+0.530284492 container died 10a77da9e004cbefc3d1bb624551d437e24d1294e06fb55594d9de2aed0c46b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-20T06:45:13.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:13 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:13.413+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:13.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:13.450+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:13.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:13.353+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:13.846 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:13 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:13.600+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:13.875 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.3.service' 2024-08-20T06:45:13.916 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:13.916 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-20T06:45:13.916 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-20T06:45:13.917 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.4 2024-08-20T06:45:14.209 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:13 smithi183 systemd[1]: Stopping Ceph osd.4 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:14.486 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:45:14.207+0000 7f2798957640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:14.486 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:45:14.207+0000 7f2798957640 -1 osd.4 76 *** Got signal Terminated *** 2024-08-20T06:45:14.486 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:45:14.207+0000 7f2798957640 -1 osd.4 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:14.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:14 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:14.396+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:14.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:14 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:14.333+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:14.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:14 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:14.420+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:14.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:14 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:14.391+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:15.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:14 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:14.638+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:15.608 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:15 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:15.293+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:15.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:15 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:15.389+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:15.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:15 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:15.383+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:15.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:15 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:15.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:15.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:15 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:15.608+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:16.583 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:16 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:16.320+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:16.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:16 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:16.409+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:16.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:16 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:16.412+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:16.805 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:16 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:16.390+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:16.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:16 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:16.582+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:17.736 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:17 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:45:17.455+0000 7f2790f64640 -1 osd.4 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:50.925327+0000 front 2024-08-20T06:44:50.925350+0000 (oldest deadline 2024-08-20T06:45:16.825476+0000) 2024-08-20T06:45:17.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:17 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:17.366+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:17.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:17 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:17.366+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:17.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:17 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:17.375+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:17.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:17 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:17.456+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:17.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:17 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:17.456+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:17.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:17 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:17.365+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:17.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:17 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:17.588+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:17.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:17 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:17.588+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:18.736 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:18 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4[33285]: 2024-08-20T06:45:18.435+0000 7f2790f64640 -1 osd.4 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:50.925327+0000 front 2024-08-20T06:44:50.925350+0000 (oldest deadline 2024-08-20T06:45:16.825476+0000) 2024-08-20T06:45:18.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:18 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:18.320+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:18.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:18 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:18.320+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:18.762 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:18 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:18.595+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:18.762 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:18 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:18.595+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:18.762 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:18 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:18.372+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:18.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:18 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:18.378+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:18.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:18 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:18.460+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:18.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:18 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:18.460+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:19.637 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:19 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:19.506+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:19.638 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:19 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:19.331+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:19.638 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:19 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:19.331+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:19.776 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:19 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:19.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:19.776 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:19 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:19.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:19.777 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:19 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:19.370+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:19.777 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:19 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:19.370+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:19.804 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:19 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:19.330+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:19.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:19 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:19.466+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:19.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:19 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:19.466+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:19.926 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:19 smithi183 podman[55889]: 2024-08-20 06:45:19.729656675 +0000 UTC m=+5.623682774 container died 247fe6ef5329d32753a62356b89469b2a4c257a67289543cf89e496ec0a53d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-20T06:45:20.188 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[55889]: 2024-08-20 06:45:20.04120149 +0000 UTC m=+5.935227588 container cleanup 247fe6ef5329d32753a62356b89469b2a4c257a67289543cf89e496ec0a53d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-20T06:45:20.189 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 bash[55889]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4 2024-08-20T06:45:20.486 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[55904]: 2024-08-20 06:45:20.309662358 +0000 UTC m=+0.582349829 container remove 247fe6ef5329d32753a62356b89469b2a4c257a67289543cf89e496ec0a53d13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-20T06:45:20.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:20 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:20.303+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:20.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:20 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:20.303+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:20.706 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:20 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:20.362+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:20.706 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:20 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:20.426+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:20.706 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:20 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:20.426+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:20.765 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:20 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:20.488+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:20.765 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[56003]: 2024-08-20 06:45:20.556259943 +0000 UTC m=+0.016750381 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:20.765 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[56003]: 2024-08-20 06:45:20.660417723 +0000 UTC m=+0.120908148 container create afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:45:20.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:20.582+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:20.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:20.582+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:20.862 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:20.381+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:20.862 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:20 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:20.381+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:21.060 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[56003]: 2024-08-20 06:45:20.862535871 +0000 UTC m=+0.323026297 container init afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD) 2024-08-20T06:45:21.060 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[56003]: 2024-08-20 06:45:20.865893704 +0000 UTC m=+0.326384131 container start afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-08-20T06:45:21.060 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:20 smithi183 podman[56003]: 2024-08-20 06:45:20.937557622 +0000 UTC m=+0.398048057 container attach afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-08-20T06:45:21.340 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:21 smithi183 podman[56003]: 2024-08-20 06:45:21.060027503 +0000 UTC m=+0.520517943 container died afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-20T06:45:21.341 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:21 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:21.261+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:21.341 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:21 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:21.261+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:21.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:21.231+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:21.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:21.344+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:21.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:21.344+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:21.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:21.456+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:21.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:21.456+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:21.592 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:21 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:21.441+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:21.592 INFO:journalctl@ceph.osd.4.smithi183.stdout:Aug 20 06:45:21 smithi183 podman[56028]: 2024-08-20 06:45:21.590181546 +0000 UTC m=+0.523626596 container remove afac6284964c939df6277d69264a1eafb77781f3cfe3c869d40a7a9d0b05e57e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-20T06:45:21.610 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.4.service' 2024-08-20T06:45:21.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:21.649 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-20T06:45:21.649 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-20T06:45:21.650 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.5 2024-08-20T06:45:21.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:21.535+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:21.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:21.535+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:21.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:21.369+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:21.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:21.369+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:21.929 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 systemd[1]: Stopping Ceph osd.5 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:22.268 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:21.928+0000 7fa9ba00f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:22.268 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:21.928+0000 7fa9ba00f640 -1 osd.5 76 *** Got signal Terminated *** 2024-08-20T06:45:22.268 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:21 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:21.928+0000 7fa9ba00f640 -1 osd.5 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:22.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:22 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:22.398+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:22.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:22 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:22.233+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:22.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:22 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:22.233+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:22.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:22 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:22.268+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:22.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:22 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:22.379+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:22.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:22 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:22.379+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:22.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:22 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:22.479+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:22.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:22 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:22.479+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:22.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:22 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:22.403+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:22.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:22 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:22.403+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:22.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:22 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:22.563+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:22.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:22 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:22.563+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:23.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:23 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:23.411+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:23.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:23 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:23.189+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:23.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:23 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:23.189+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:23.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:23 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:23.290+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:23.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:23 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:23.399+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:23.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:23 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:23.399+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:23.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:23 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:23.484+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:23.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:23 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:23.484+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:23.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:23 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:23.387+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:23.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:23 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:23.387+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:23.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:23 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:23.609+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:23.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:23 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:23.609+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:24.438 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:24 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:24.158+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:24.438 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:24 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:24.158+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:24.524 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:24 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:24.268+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:24.524 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:24 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:24.376+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:24.524 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:24 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:24.376+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:24.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:24 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:24.436+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:24.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:24 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:24.523+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:24.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:24 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:24.523+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:24.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:24.599+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:24.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:24.599+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:24.862 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:24.431+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:24.862 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:24 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:24.431+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:25.436 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:25.133+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:25.437 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:25.133+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:25.437 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:25.133+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:25.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:25.245+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:25.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:25.378+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:25.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:25.378+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:25.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:25.435+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:25.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:25 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:25.435+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:25.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:25 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:25.614+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:25.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:25 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:25.614+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:25.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:25 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:25.393+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:25.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:25 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:25.393+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:26.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:25.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:26.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:25.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:26.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:25 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:25.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:26.426 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:26.173+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:26.426 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:26.173+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:26.426 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:26.173+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:26.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:26.276+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:26.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:26.331+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:45.429729+0000 front 2024-08-20T06:44:45.429648+0000 (oldest deadline 2024-08-20T06:45:10.129702+0000) 2024-08-20T06:45:26.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:26.331+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.830735+0000 front 2024-08-20T06:44:54.830805+0000 (oldest deadline 2024-08-20T06:45:20.730838+0000) 2024-08-20T06:45:26.555 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5[33298]: 2024-08-20T06:45:26.331+0000 7fa9b6624640 -1 osd.5 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.731300+0000 front 2024-08-20T06:45:00.731342+0000 (oldest deadline 2024-08-20T06:45:26.031458+0000) 2024-08-20T06:45:26.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:26.354+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:26.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:26.354+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:26.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:26.424+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:26.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:26 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:26.424+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:27.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:26.584+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:27.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:26.584+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:27.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:26 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:26.584+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:27.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:26.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:27.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:26.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:27.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:27 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:27.436+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:27.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:27 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:27.436+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:27.490 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:27 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:27.213+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:27.490 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:27 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:27.213+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:27.490 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:27 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:27.213+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:27.554 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:27 smithi193 podman[57203]: 2024-08-20 06:45:27.35080662 +0000 UTC m=+5.503148575 container died 221848fc596b05cde76bec403050cafca078bd341765fb4b1a8d303200b924ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-20T06:45:27.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:27 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:27.301+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:27.620 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:27 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:27.367+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:27.621 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:27 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:27.367+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:27.988 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:27 smithi193 podman[57203]: 2024-08-20 06:45:27.711137063 +0000 UTC m=+5.863479007 container cleanup 221848fc596b05cde76bec403050cafca078bd341765fb4b1a8d303200b924ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-20T06:45:27.988 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:27 smithi193 bash[57203]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5 2024-08-20T06:45:27.988 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:27 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:27.567+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:27.988 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:27 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:27.567+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:27.988 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:27 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:27.567+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:28.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:27 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:27.620+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:28.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:27 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:27.620+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:28.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:27 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:27.620+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:28.265 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:27 smithi193 podman[57221]: 2024-08-20 06:45:27.987428491 +0000 UTC m=+0.629209303 container remove 221848fc596b05cde76bec403050cafca078bd341765fb4b1a8d303200b924ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-08-20T06:45:28.445 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:28.183+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:28.445 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:28.183+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:28.445 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:28.183+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:28.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:28.264+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:28.535 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.235244311 +0000 UTC m=+0.016159541 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:28.535 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.355444871 +0000 UTC m=+0.136360089 container create 66d98976cc7e9c60a7b0819c6cef20aa43895f6bf6cec333b5d90e0103b1c3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:45:28.535 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.532774621 +0000 UTC m=+0.313689841 container init 66d98976cc7e9c60a7b0819c6cef20aa43895f6bf6cec333b5d90e0103b1c3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:45:28.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:28.330+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:28.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:28.330+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:28.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:28.330+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:28.738 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:28.444+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:28.738 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:28 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:28.444+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:28.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:28.601+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:28.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:28.601+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:28.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:28 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:28.601+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:28.805 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.536329052 +0000 UTC m=+0.317244284 container start 66d98976cc7e9c60a7b0819c6cef20aa43895f6bf6cec333b5d90e0103b1c3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T06:45:28.805 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.624505608 +0000 UTC m=+0.405420832 container attach 66d98976cc7e9c60a7b0819c6cef20aa43895f6bf6cec333b5d90e0103b1c3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-08-20T06:45:28.805 INFO:journalctl@ceph.osd.5.smithi193.stdout:Aug 20 06:45:28 smithi193 podman[57308]: 2024-08-20 06:45:28.720439624 +0000 UTC m=+0.501354854 container died 66d98976cc7e9c60a7b0819c6cef20aa43895f6bf6cec333b5d90e0103b1c3da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-20T06:45:29.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:28.650+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:29.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:28.650+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:29.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:28 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:28.650+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:29.280 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.5.service' 2024-08-20T06:45:29.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:29.320 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-20T06:45:29.320 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-20T06:45:29.320 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.6 2024-08-20T06:45:29.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:29 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:29.154+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:29.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:29 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:29.154+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:29.447 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:29 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:29.154+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:29.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:29 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:29.311+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:29.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:29 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:29.311+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.304+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.304+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.304+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 systemd[1]: Stopping Ceph osd.6 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.605+0000 7f5dcc37d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.605+0000 7f5dcc37d640 -1 osd.6 76 *** Got signal Terminated *** 2024-08-20T06:45:29.607 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:29.605+0000 7f5dcc37d640 -1 osd.6 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:29.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:29 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:29.446+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:29.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:29 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:29.446+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:29.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:29 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:29.552+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:29.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:29 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:29.552+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:29.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:29 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:29.552+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:29.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:29.645+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:29.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:29.645+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:29.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:29.645+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:30.485 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:30 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:30.123+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:30.485 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:30 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:30.123+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:30.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:30 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:30.123+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:30.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:30.296+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:30.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:30.296+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:30.612 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:30.296+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:30.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:30 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:30.484+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:30.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:30 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:30.484+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:30.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:30 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:30.334+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:30.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:30 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:30.334+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:30.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:30 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:30.514+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:30.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:30 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:30.514+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:30.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:30 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:30.514+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:31.050 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:30.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:31.050 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:30.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:31.050 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:30 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:30.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:31.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:31.081+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:31.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:31.081+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:31.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:31.081+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:31.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:31.325+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:31.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:31.325+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:31.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:31.325+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:31.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:31 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:31.358+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:31.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:31 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:31.358+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:31.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:31 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:31.521+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:31.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:31 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:31.521+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:31.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:31 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:31.521+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:31.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:31.496+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:31.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:31 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:31.496+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:32.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:31.618+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:32.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:31.618+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:32.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:31 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:31.618+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:32.478 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:32 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:32.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:32.479 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:32 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:32.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:32.479 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:32 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:32.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:32.529 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:32 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:32.354+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:32.530 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:32 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:32.354+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:32.530 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:32 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:32.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:32.530 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:32 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:32.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:32.530 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:32 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:32.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:32.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:32.339+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:32.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:32.339+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:32.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:32.339+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:32.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:32 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:32.477+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:32.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:32 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:32.477+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:33.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:32.627+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:33.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:32.627+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:33.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:32.627+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:33.112 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:32 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:32.627+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:33.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:33.031+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:33.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:33.031+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:33.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:33.031+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:33.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:33.338+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:33.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:33.338+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:33.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:33.338+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:33.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:33.325+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:33.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:33.325+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:33.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:33.325+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:33.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:33.505+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:33.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:33.505+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:33.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:33.505+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:33.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:33 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:33.505+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:33.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:33.485+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:33.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:33.485+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:33.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:33 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:33.485+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:34.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:33.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:34.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:33.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:34.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:33.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:34.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:33 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:33.610+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:34.449 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:34.070+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:34.449 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:34.070+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:34.449 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:34.070+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:34.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:34.344+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:43.990325+0000 front 2024-08-20T06:44:43.990354+0000 (oldest deadline 2024-08-20T06:45:09.290439+0000) 2024-08-20T06:45:34.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:34.344+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:55.191364+0000 front 2024-08-20T06:44:55.191444+0000 (oldest deadline 2024-08-20T06:45:19.291492+0000) 2024-08-20T06:45:34.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6[45951]: 2024-08-20T06:45:34.344+0000 7f5dc8992640 -1 osd.6 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:03.292901+0000 front 2024-08-20T06:45:03.292849+0000 (oldest deadline 2024-08-20T06:45:27.392907+0000) 2024-08-20T06:45:34.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:34.448+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:34.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:34.448+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:34.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:34.448+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:34.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:34.356+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:34.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:34.356+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:34.804 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:34.356+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:34.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:34.524+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:34.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:34.524+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:34.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:34.524+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:34.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:34 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:34.524+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:35.053 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:34.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:35.054 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:34.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:35.054 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:34.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:35.054 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:34 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:34.631+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:35.339 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:35 smithi107 podman[332102]: 2024-08-20 06:45:35.053305069 +0000 UTC m=+5.531529081 container died 192a52db8b564ee60c9c5a378dc3c9df348fd21efac6fde3ca84cc8034c31a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-20T06:45:35.403 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:35.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:35.403 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:35.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:35.403 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:35.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:35.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:35 smithi107 podman[332102]: 2024-08-20 06:45:35.539045371 +0000 UTC m=+6.017269376 container cleanup 192a52db8b564ee60c9c5a378dc3c9df348fd21efac6fde3ca84cc8034c31a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-20T06:45:35.611 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:35 smithi107 bash[332102]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6 2024-08-20T06:45:35.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:35.402+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:35.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:35.402+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:35.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:35 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:35.402+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:35.317+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:35.317+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:35.317+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:35.525+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:35.525+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:35.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:35.525+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:35.806 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:35 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:35.525+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:36.111 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:35 smithi107 podman[332116]: 2024-08-20 06:45:35.832375158 +0000 UTC m=+0.772782868 container remove 192a52db8b564ee60c9c5a378dc3c9df348fd21efac6fde3ca84cc8034c31a43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:36.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:35.647+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:36.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:35.647+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:36.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:35.647+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:36.111 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:35 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:35.647+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:36.388 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:36.388 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:36.388 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:36.478 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.059869624 +0000 UTC m=+0.015912267 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:36.478 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.174736196 +0000 UTC m=+0.130778841 container create 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:45:36.478 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.40255987 +0000 UTC m=+0.358602514 container init 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:45:36.478 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.405748391 +0000 UTC m=+0.361791035 container start 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-20T06:45:36.478 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.477367598 +0000 UTC m=+0.433410261 container attach 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:36.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:36.302+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:36.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:36.302+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:36.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:36.302+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:36.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:36.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:36.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:36.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:36.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:36.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:36.861 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:36 smithi107 podman[332207]: 2024-08-20 06:45:36.591489077 +0000 UTC m=+0.547531722 container died 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T06:45:36.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:36 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:36.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:36.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:36 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:36.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:36.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:36 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:36.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:36.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:36 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:36.611+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:37.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:36.556+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:37.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:36.556+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:37.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:36.556+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:37.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:36 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:36.556+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:37.134 INFO:journalctl@ceph.osd.6.smithi107.stdout:Aug 20 06:45:37 smithi107 podman[332231]: 2024-08-20 06:45:37.131410502 +0000 UTC m=+0.533884319 container remove 1e4e63f6afa12daa20733555b7def6502ff814dce984964594ae0094beb84a91 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-20T06:45:37.153 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.6.service' 2024-08-20T06:45:37.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:37.193 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-20T06:45:37.193 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-20T06:45:37.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.7 2024-08-20T06:45:37.221 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.964+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:37.221 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.964+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:37.221 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:36 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:36.964+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:37.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 systemd[1]: Stopping Ceph osd.7 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:37.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.405+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:37.486 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.405+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:37.487 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.405+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:37.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:37.543+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:37.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:37.543+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:37.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:37.543+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:37.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:37.543+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:37.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:37.315+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:37.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:37.315+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:37.805 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:37 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:37.315+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:37.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:37 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:37.584+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:37.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:37 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:37.584+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:37.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:37 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:37.584+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:37.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:37 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:37.584+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:37.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.492+0000 7f603cde2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:37.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.492+0000 7f603cde2640 -1 osd.7 76 *** Got signal Terminated *** 2024-08-20T06:45:37.986 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:37.492+0000 7f603cde2640 -1 osd.7 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:38.379 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.006+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:38.379 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.006+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:38.379 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.006+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:38.380 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.006+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:38.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:38.288+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:38.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:38.288+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:38.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:38.288+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:38.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:38.378+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:38.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:38.378+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:38.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:38.378+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:38.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:38 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:38.605+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:38.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:38 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:38.605+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:38.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:38 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:38.605+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:38.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:38 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:38.605+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:39.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:38.563+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:39.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:38.563+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:39.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:38.563+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:39.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:38 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:38.563+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:39.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.974+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:39.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.974+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:39.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.974+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:39.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:38 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:38.974+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:39.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:39.264+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:39.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:39.264+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:39.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:39.264+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:39.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:39.355+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:39.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:39.355+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:39.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:39.355+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:39.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:39 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:39.561+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:39.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:39 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:39.561+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:39.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:39 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:39.561+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:39.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:39 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:39.561+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:40.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:39.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:40.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:39.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:40.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:39.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:40.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:39 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:39.558+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:40.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:39.981+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:40.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:39.981+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:40.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:39.981+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:40.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:39 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:39.981+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:40.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:40.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:40.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:40.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:40.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:40.526+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:40.526+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:40.526+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:40.526+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:40.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:40 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:40.526+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:40.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:40 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:40.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:40.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:40 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:40.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:40.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:40 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:40.387+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:40.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:40 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:40.516+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:40.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:40 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:40.516+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:40.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:40 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:40.516+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:40.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:40 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:40.516+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:41.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:41.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:41.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:41.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:41.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:41.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:41.374 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:41.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:41.501 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:41.234+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:41.501 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:41.234+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:41.501 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:41.234+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:41.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:41.373+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:41.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:41.373+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:41.737 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:41.373+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:41.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:41.500+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:41.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:41.500+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:41.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:41.500+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:41.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:41.500+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:41.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:41 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:41.500+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:41.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:41.495+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:41.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:41.495+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:41.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:41.495+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:41.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:41 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:41.495+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:42.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:42.035+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:42.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:42.035+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:42.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:42.035+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:42.385 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:42.035+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:42.523 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:42.196+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:42.523 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:42.196+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:42.523 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:42.196+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:42.523 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:42.196+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:42.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:42.383+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.973173+0000 front 2024-08-20T06:44:52.973193+0000 (oldest deadline 2024-08-20T06:45:18.873511+0000) 2024-08-20T06:45:42.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:42.383+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:58.874307+0000 front 2024-08-20T06:44:58.874274+0000 (oldest deadline 2024-08-20T06:45:24.774112+0000) 2024-08-20T06:45:42.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:42.383+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:08.775864+0000 front 2024-08-20T06:45:08.775852+0000 (oldest deadline 2024-08-20T06:45:33.376654+0000) 2024-08-20T06:45:42.736 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:42 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7[39763]: 2024-08-20T06:45:42.383+0000 7f60353ef640 -1 osd.7 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:18.677485+0000 front 2024-08-20T06:45:18.677534+0000 (oldest deadline 2024-08-20T06:45:41.577572+0000) 2024-08-20T06:45:42.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:42.522+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:42.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:42.522+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:42.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:42.522+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:42.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:42.522+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:42.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:42 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:42.522+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:42.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:42.499+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:42.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:42.499+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:42.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:42.499+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:42.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:42.499+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:42.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:42 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:42.499+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:43.268 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 podman[56110]: 2024-08-20 06:45:43.009360893 +0000 UTC m=+5.625123814 container died 81301b6d383bf384bb6ea02215ea6ceaf9b55e73d3128329a786571229c38b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-20T06:45:43.268 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:43.268 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:43.269 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:43.269 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:43.269 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:43.540 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:43.186+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:43.540 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:43.186+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:43.540 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:43.186+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:43.540 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:43.186+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:43.643 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 podman[56110]: 2024-08-20 06:45:43.367511289 +0000 UTC m=+5.983274206 container cleanup 81301b6d383bf384bb6ea02215ea6ceaf9b55e73d3128329a786571229c38b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:45:43.643 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 bash[56110]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7 2024-08-20T06:45:43.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:43.539+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:43.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:43.539+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:43.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:43.539+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:43.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:43.539+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:43.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:43 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:43.539+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:43.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:43 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:43.465+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:43.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:43 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:43.465+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:43.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:43 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:43.465+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:43.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:43 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:43.465+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:43.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:43 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:43.465+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:43.969 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 podman[56126]: 2024-08-20 06:45:43.643507711 +0000 UTC m=+0.627387638 container remove 81301b6d383bf384bb6ea02215ea6ceaf9b55e73d3128329a786571229c38b89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.980+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.980+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.980+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.980+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:43 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:43.980+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:44.236 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 podman[56214]: 2024-08-20 06:45:43.969209636 +0000 UTC m=+0.110029523 container create 4f9ae8db519ec066e2999552f7dc064e86939d92b8a1dd26a936ca5ff2d39f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS) 2024-08-20T06:45:44.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:43 smithi183 podman[56214]: 2024-08-20 06:45:43.875326217 +0000 UTC m=+0.016146119 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:44.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:44 smithi183 podman[56214]: 2024-08-20 06:45:44.205523991 +0000 UTC m=+0.346343877 container init 4f9ae8db519ec066e2999552f7dc064e86939d92b8a1dd26a936ca5ff2d39f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731) 2024-08-20T06:45:44.237 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:44 smithi183 podman[56214]: 2024-08-20 06:45:44.208759878 +0000 UTC m=+0.349579773 container start 4f9ae8db519ec066e2999552f7dc064e86939d92b8a1dd26a936ca5ff2d39f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:45:44.529 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:44.223+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:44.529 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:44.223+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:44.529 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:44.223+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:44.529 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:44.223+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:44.700 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:44 smithi183 podman[56214]: 2024-08-20 06:45:44.272273868 +0000 UTC m=+0.413093767 container attach 4f9ae8db519ec066e2999552f7dc064e86939d92b8a1dd26a936ca5ff2d39f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:44.700 INFO:journalctl@ceph.osd.7.smithi183.stdout:Aug 20 06:45:44 smithi183 podman[56214]: 2024-08-20 06:45:44.39059367 +0000 UTC m=+0.531413559 container died 4f9ae8db519ec066e2999552f7dc064e86939d92b8a1dd26a936ca5ff2d39f82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:45:44.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:44.528+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:44.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:44.528+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:44.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:44.528+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:44.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:44.528+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:44.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:44 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:44.528+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:44.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:44.441+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:44.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:44.441+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:44.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:44.441+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:44.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:44.441+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:44.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:44 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:44.441+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:45.000 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.7.service' 2024-08-20T06:45:45.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:45.080 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-20T06:45:45.080 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-20T06:45:45.080 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.8 2024-08-20T06:45:45.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:44.954+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:45.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:44.954+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:45.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:44.954+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:45.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:44.954+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:45.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:44 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:44.954+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 systemd[1]: Stopping Ceph osd.8 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.242+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.242+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.242+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.242+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:45.372 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.370+0000 7f322242a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:45.373 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.370+0000 7f322242a640 -1 osd.8 76 *** Got signal Terminated *** 2024-08-20T06:45:45.373 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:45.370+0000 7f322242a640 -1 osd.8 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:45.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:45.504+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:45.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:45.504+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:45.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:45.504+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:45.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:45.504+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:45.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:45 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:45.504+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:45.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:45 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:45.478+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:45.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:45 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:45.478+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:45.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:45 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:45.478+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:45.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:45 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:45.478+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:45.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:45 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:45.478+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:46.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:45.997+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:46.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:45.997+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:46.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:45.997+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:46.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:45.997+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:46.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:45.997+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:46.549 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:46.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:46.549 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:46.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:46.549 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:46.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:46.549 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:46.278+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:46.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:46.548+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:46.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:46.548+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:46.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:46.548+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:46.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:46.548+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:46.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:46 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:46.548+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:46.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:46 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:46.467+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:46.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:46 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:46.467+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:46.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:46 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:46.467+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:46.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:46 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:46.467+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:46.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:46 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:46.467+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:47.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.021+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:47.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.021+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:47.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.021+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:47.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.021+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:47.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.021+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:47.534 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:47.261+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:47.534 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:47.261+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:47.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:47.261+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:47.535 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:47.261+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:47.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:47.533+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:47.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:47.533+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:47.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:47.533+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:47.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:47.533+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:47.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:47.533+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:47.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:47.436+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:47.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:47.436+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:47.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:47.436+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:47.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:47.436+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:47.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:47 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:47.436+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:48.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.978+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:48.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.978+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:48.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.978+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:48.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.978+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:48.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:47 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:47.978+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:48.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:48.303+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:48.303+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:48.303+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:48.303+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:48.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:48 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:48.527+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:48.763 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:48 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:48.417+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:49.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:49 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:49.002+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:49.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:49 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:49.002+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:49.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:49 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:49.002+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:49.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:49 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:49.002+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:49.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:49 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:49.002+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:49.256+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:49.256+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:49.256+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:49.256+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:49.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:49.556 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:49 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:49.487+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:49.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:49.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:49.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:49.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:49.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:49.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:49 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:49.420+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:50.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:50.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:50.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:50.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:50.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:50.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:50 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:50.050+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:50.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:50.296+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:54.436748+0000 front 2024-08-20T06:44:54.436565+0000 (oldest deadline 2024-08-20T06:45:20.336502+0000) 2024-08-20T06:45:50.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:50.296+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:00.337119+0000 front 2024-08-20T06:45:00.337147+0000 (oldest deadline 2024-08-20T06:45:29.137793+0000) 2024-08-20T06:45:50.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:50.296+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.438803+0000 front 2024-08-20T06:45:11.438723+0000 (oldest deadline 2024-08-20T06:45:32.538690+0000) 2024-08-20T06:45:50.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8[40205]: 2024-08-20T06:45:50.296+0000 7f321aa37640 -1 osd.8 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.940048+0000 front 2024-08-20T06:45:15.940062+0000 (oldest deadline 2024-08-20T06:45:41.839934+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:50.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:50 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:50.475+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:50.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:50.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:50.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:50.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:50.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:50.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:50 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:50.446+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:51.030 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:50 smithi193 podman[57409]: 2024-08-20 06:45:50.774151731 +0000 UTC m=+5.497896988 container died 7faf500d55d03c6f5a179db8b00310555ad298aa5b6aa9d27f0c5600cf1052aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:45:51.299 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:51 smithi193 podman[57409]: 2024-08-20 06:45:51.112824837 +0000 UTC m=+5.836570092 container cleanup 7faf500d55d03c6f5a179db8b00310555ad298aa5b6aa9d27f0c5600cf1052aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0) 2024-08-20T06:45:51.300 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:51 smithi193 bash[57409]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8 2024-08-20T06:45:51.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:51.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:51.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:51.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:51.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:51.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:51 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:51.073+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:51.554 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:51 smithi193 podman[57425]: 2024-08-20 06:45:51.299768376 +0000 UTC m=+0.524497536 container remove 7faf500d55d03c6f5a179db8b00310555ad298aa5b6aa9d27f0c5600cf1052aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0) 2024-08-20T06:45:51.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:51.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:51.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:51.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:51.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:51.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:51 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:51.452+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:51.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:51.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:51.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:51.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:51.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:51.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:51 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:51.412+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:52.055 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:51 smithi193 podman[57523]: 2024-08-20 06:45:51.670717764 +0000 UTC m=+0.022914480 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:52.055 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:51 smithi193 podman[57523]: 2024-08-20 06:45:51.773098325 +0000 UTC m=+0.125295032 container create 2f35f3ea6e0ee45431dd46e03c3e28a0cd53dcb6e85d1ac555a093e1eced8b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0) 2024-08-20T06:45:52.055 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:52 smithi193 podman[57523]: 2024-08-20 06:45:52.000156359 +0000 UTC m=+0.352353074 container init 2f35f3ea6e0ee45431dd46e03c3e28a0cd53dcb6e85d1ac555a093e1eced8b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-20T06:45:52.055 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:52 smithi193 podman[57523]: 2024-08-20 06:45:52.003444358 +0000 UTC m=+0.355641072 container start 2f35f3ea6e0ee45431dd46e03c3e28a0cd53dcb6e85d1ac555a093e1eced8b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-20T06:45:52.438 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:52 smithi193 podman[57523]: 2024-08-20 06:45:52.094022857 +0000 UTC m=+0.446219572 container attach 2f35f3ea6e0ee45431dd46e03c3e28a0cd53dcb6e85d1ac555a093e1eced8b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T06:45:52.438 INFO:journalctl@ceph.osd.8.smithi193.stdout:Aug 20 06:45:52 smithi193 podman[57523]: 2024-08-20 06:45:52.197436612 +0000 UTC m=+0.549633327 container died 2f35f3ea6e0ee45431dd46e03c3e28a0cd53dcb6e85d1ac555a093e1eced8b4a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T06:45:52.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:52.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:52.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:52.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:52.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:52.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:52 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:52.064+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:52.788 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:52.437+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:52.809 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.8.service' 2024-08-20T06:45:52.847 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:45:52.848 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-20T06:45:52.848 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-20T06:45:52.848 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.9 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:52.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:52.396+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:53.150 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:52 smithi107 systemd[1]: Stopping Ceph osd.9 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:53.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:53 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:53.054+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.148+0000 7fdcfac25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.148+0000 7fdcfac25640 -1 osd.9 76 *** Got signal Terminated *** 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.148+0000 7fdcfac25640 -1 osd.9 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:53.611 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:53.612 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:53.612 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:53.612 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:53 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:53.368+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:53.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:53.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:53.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:53.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:53.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:53.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:53 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:53.411+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:54.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:54 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:54.009+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:54.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:54.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:54.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:54.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:54.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:54.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:54 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:54.375+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:54.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:54.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:54.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:54.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:54.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:54.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:54 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:54.403+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:55.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:55.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:55.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:55.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:55.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:55.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.000+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:55.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:55.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:55.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:55.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:55.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:55.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:55 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:55.402+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:55.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:55 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:55.398+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:56.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:56.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:56.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:56.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:56.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:56.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:56.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:55 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:55.959+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:45:56.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:56.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:56.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:56.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:56.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:56.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:56 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:56.408+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:56.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:56.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:56.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:56.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:56.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:56.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:56.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:56 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:56.445+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:33.497436+0000 front 2024-08-20T06:45:33.497462+0000 (oldest deadline 2024-08-20T06:45:55.797522+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:57.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:57.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:56 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:56.986+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:45:57.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:57.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:57.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:57.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:57.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:57.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:57.385+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:57.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:46.191190+0000 front 2024-08-20T06:44:46.191153+0000 (oldest deadline 2024-08-20T06:45:10.891328+0000) 2024-08-20T06:45:57.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.192235+0000 front 2024-08-20T06:44:53.192278+0000 (oldest deadline 2024-08-20T06:45:17.292441+0000) 2024-08-20T06:45:57.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:01.893743+0000 front 2024-08-20T06:45:01.893843+0000 (oldest deadline 2024-08-20T06:45:27.193641+0000) 2024-08-20T06:45:57.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:10.694750+0000 front 2024-08-20T06:45:10.694688+0000 (oldest deadline 2024-08-20T06:45:32.394791+0000) 2024-08-20T06:45:57.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.695809+0000 front 2024-08-20T06:45:17.695944+0000 (oldest deadline 2024-08-20T06:45:42.395740+0000) 2024-08-20T06:45:57.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.396507+0000 front 2024-08-20T06:45:22.396475+0000 (oldest deadline 2024-08-20T06:45:48.296314+0000) 2024-08-20T06:45:57.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:57 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9[52486]: 2024-08-20T06:45:57.464+0000 7fdcf3232640 -1 osd.9 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:33.497436+0000 front 2024-08-20T06:45:33.497462+0000 (oldest deadline 2024-08-20T06:45:55.797522+0000) 2024-08-20T06:45:58.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.032+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:58.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:58.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:58.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:58.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:58.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:58.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:58 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:58.033+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:45:58.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:58.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:58.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:58.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:58.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:58.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:58 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:58.398+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:58.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:58 smithi107 podman[332309]: 2024-08-20 06:45:58.605926427 +0000 UTC m=+5.564881097 container died e53995f2779750a927dbffcc5782b13734748b4d844954212f929714163e5908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:45:59.267 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:58 smithi107 podman[332309]: 2024-08-20 06:45:58.95672177 +0000 UTC m=+5.915676432 container cleanup e53995f2779750a927dbffcc5782b13734748b4d844954212f929714163e5908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:45:59.267 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:58 smithi107 bash[332309]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9 2024-08-20T06:45:59.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:45:59.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:45:59.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:45:59.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:45:59.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:45:59.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:45:59.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:45:59 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:45:59.078+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:45:59.609 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332325]: 2024-08-20 06:45:59.266592208 +0000 UTC m=+0.653762229 container remove e53995f2779750a927dbffcc5782b13734748b4d844954212f929714163e5908 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-20T06:45:59.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:45:59.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:45:59.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:45:59.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:45:59.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:45:59.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:45:59 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:45:59.371+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:45:59.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332416]: 2024-08-20 06:45:59.609195728 +0000 UTC m=+0.106179213 container create ec67d4df8f5762f601eaa53430291dc81c57967c9a326d05fee889bc8c526885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-08-20T06:45:59.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332416]: 2024-08-20 06:45:59.518899692 +0000 UTC m=+0.015883156 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:45:59.861 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332416]: 2024-08-20 06:45:59.811596998 +0000 UTC m=+0.308580441 container init ec67d4df8f5762f601eaa53430291dc81c57967c9a326d05fee889bc8c526885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True) 2024-08-20T06:45:59.862 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332416]: 2024-08-20 06:45:59.814794391 +0000 UTC m=+0.311777842 container start ec67d4df8f5762f601eaa53430291dc81c57967c9a326d05fee889bc8c526885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-20T06:46:00.298 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:45:59 smithi107 podman[332416]: 2024-08-20 06:45:59.895155306 +0000 UTC m=+0.392138755 container attach ec67d4df8f5762f601eaa53430291dc81c57967c9a326d05fee889bc8c526885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-20T06:46:00.298 INFO:journalctl@ceph.osd.9.smithi107.stdout:Aug 20 06:46:00 smithi107 podman[332416]: 2024-08-20 06:46:00.00814143 +0000 UTC m=+0.505124884 container died ec67d4df8f5762f601eaa53430291dc81c57967c9a326d05fee889bc8c526885 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-20T06:46:00.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:00.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:00.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:00.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:00.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:00.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:00.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.058+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:00.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.9.service' 2024-08-20T06:46:00.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:00.658 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-20T06:46:00.658 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-20T06:46:00.658 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.10 2024-08-20T06:46:00.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:00.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:00.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:00.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:00.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:00.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:00 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:00.406+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:00.970 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 systemd[1]: Stopping Ceph osd.10 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:01.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.969+0000 7f3127007640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:01.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.969+0000 7f3127007640 -1 osd.10 76 *** Got signal Terminated *** 2024-08-20T06:46:01.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:00 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:00.969+0000 7f3127007640 -1 osd.10 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:46:01.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:01.237 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:01 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:01.061+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:01.716 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:01.363+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:02.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:02.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:02.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:02.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:02.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:02.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:02.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:02 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:02.044+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:02.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:02.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:02.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:02.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:02.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:02.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:02 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:02.322+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:03.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:03.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:03.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:03.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:03.488 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:03.488 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:03.488 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:03 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:03.037+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:03.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:03.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:03.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:03.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:03.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:03.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:03 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:03.316+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:04.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:04.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:04.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:04.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:04.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:04.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:04.487 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:04 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:04.001+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:04.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:04.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:04.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:04.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:04.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:04.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:04 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:04.354+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:05.029 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:42.731022+0000 front 2024-08-20T06:44:42.731071+0000 (oldest deadline 2024-08-20T06:45:12.731385+0000) 2024-08-20T06:46:05.029 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:52.731843+0000 front 2024-08-20T06:44:52.731813+0000 (oldest deadline 2024-08-20T06:45:17.332506+0000) 2024-08-20T06:46:05.029 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:45:02.033664+0000 front 2024-08-20T06:45:02.033799+0000 (oldest deadline 2024-08-20T06:45:24.933752+0000) 2024-08-20T06:46:05.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.235918+0000 front 2024-08-20T06:45:11.235998+0000 (oldest deadline 2024-08-20T06:45:37.135992+0000) 2024-08-20T06:46:05.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:17.136444+0000 front 2024-08-20T06:45:17.136419+0000 (oldest deadline 2024-08-20T06:45:42.436555+0000) 2024-08-20T06:46:05.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:22.437426+0000 front 2024-08-20T06:45:22.437297+0000 (oldest deadline 2024-08-20T06:45:50.037544+0000) 2024-08-20T06:46:05.486 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:05 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10[46306]: 2024-08-20T06:46:05.026+0000 7f312361c640 -1 osd.10 76 heartbeat_check: no reply from 172.21.15.107:6822 osd.6 since back 2024-08-20T06:45:30.038040+0000 front 2024-08-20T06:45:30.037960+0000 (oldest deadline 2024-08-20T06:45:55.937922+0000) 2024-08-20T06:46:05.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:05.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:05.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:05.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:05.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:05.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:05.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:05 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:05.395+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:06.736 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:06 smithi183 podman[56330]: 2024-08-20 06:46:06.476066825 +0000 UTC m=+5.625169155 container died ca33e163b5d67d01d649369db83991c4d06c629a929006745751db93b59dc6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-08-20T06:46:06.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:06.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:06 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:06.351+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:07.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:06 smithi183 podman[56330]: 2024-08-20 06:46:06.793750483 +0000 UTC m=+5.942852810 container cleanup ca33e163b5d67d01d649369db83991c4d06c629a929006745751db93b59dc6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-20T06:46:07.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:06 smithi183 bash[56330]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10 2024-08-20T06:46:07.236 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:06 smithi183 podman[56345]: 2024-08-20 06:46:06.986676323 +0000 UTC m=+0.504550518 container remove ca33e163b5d67d01d649369db83991c4d06c629a929006745751db93b59dc6c3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:46:07.681 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.395628797 +0000 UTC m=+0.104612981 container create 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:46:07.682 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.306477647 +0000 UTC m=+0.015461840 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:46:07.682 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.597835228 +0000 UTC m=+0.306819416 container init 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-08-20T06:46:07.682 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.601039601 +0000 UTC m=+0.310023786 container start 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T06:46:07.682 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.681058234 +0000 UTC m=+0.390042424 container attach 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-20T06:46:07.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:07.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:07 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:07.315+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:07.986 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:07 smithi183 podman[56434]: 2024-08-20 06:46:07.786733599 +0000 UTC m=+0.495717784 container died 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:46:08.304 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.10.service' 2024-08-20T06:46:08.327 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:08 smithi183 podman[56457]: 2024-08-20 06:46:08.28437742 +0000 UTC m=+0.491818632 container remove 8f3014bf99166fc1fe333e2292f3256bd5d0b7596622177532363419cbf9963d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-10-deactivate, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:46:08.327 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:08 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.10.service: Deactivated successfully. 2024-08-20T06:46:08.328 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:08 smithi183 systemd[1]: Stopped Ceph osd.10 for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:46:08.328 INFO:journalctl@ceph.osd.10.smithi183.stdout:Aug 20 06:46:08 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.10.service: Consumed 28.691s CPU time. 2024-08-20T06:46:08.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:08.342 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-20T06:46:08.343 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-20T06:46:08.343 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.11 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.272+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:08.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 systemd[1]: Stopping Ceph osd.11 for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:09.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.631+0000 7f78f3b57640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:09.054 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.631+0000 7f78f3b57640 -1 osd.11 76 *** Got signal Terminated *** 2024-08-20T06:46:09.055 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:08 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:08.631+0000 7f78f3b57640 -1 osd.11 76 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T06:46:09.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:09.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:09.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:09.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:09.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:09.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:09.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:09 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:09.299+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:10.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:10.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:10 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:10.321+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:11.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:11.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:11.804 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:11.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:11.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:11.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:11.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:11 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:11.320+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:12.805 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:12 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:12.331+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:13.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-08-20T06:44:44.538610+0000 front 2024-08-20T06:44:44.538581+0000 (oldest deadline 2024-08-20T06:45:09.238530+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6804 osd.1 since back 2024-08-20T06:44:53.840151+0000 front 2024-08-20T06:44:53.840245+0000 (oldest deadline 2024-08-20T06:45:17.340077+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6804 osd.2 since back 2024-08-20T06:44:59.640831+0000 front 2024-08-20T06:44:59.640882+0000 (oldest deadline 2024-08-20T06:45:24.940972+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.107:6814 osd.3 since back 2024-08-20T06:45:11.242677+0000 front 2024-08-20T06:45:11.242753+0000 (oldest deadline 2024-08-20T06:45:32.942627+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6812 osd.4 since back 2024-08-20T06:45:15.843498+0000 front 2024-08-20T06:45:15.843580+0000 (oldest deadline 2024-08-20T06:45:39.943519+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6812 osd.5 since back 2024-08-20T06:45:25.144669+0000 front 2024-08-20T06:45:25.144755+0000 (oldest deadline 2024-08-20T06:45:48.044728+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.183:6820 osd.7 since back 2024-08-20T06:45:42.448253+0000 front 2024-08-20T06:45:42.448163+0000 (oldest deadline 2024-08-20T06:46:04.748024+0000) 2024-08-20T06:46:13.555 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:13 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11[47187]: 2024-08-20T06:46:13.297+0000 7f78f016c640 -1 osd.11 76 heartbeat_check: no reply from 172.21.15.193:6820 osd.8 since back 2024-08-20T06:45:47.648843+0000 front 2024-08-20T06:45:47.648863+0000 (oldest deadline 2024-08-20T06:46:12.849438+0000) 2024-08-20T06:46:14.304 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:14 smithi193 podman[57629]: 2024-08-20 06:46:14.049740347 +0000 UTC m=+5.508572480 container died 301c04c6cd0244d4f90a6821ca8aa67cc428a006e4c07e889c3c544af2205000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0) 2024-08-20T06:46:14.674 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:14 smithi193 podman[57629]: 2024-08-20 06:46:14.414294116 +0000 UTC m=+5.873126241 container cleanup 301c04c6cd0244d4f90a6821ca8aa67cc428a006e4c07e889c3c544af2205000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-08-20T06:46:14.674 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:14 smithi193 bash[57629]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11 2024-08-20T06:46:15.017 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:14 smithi193 podman[57645]: 2024-08-20 06:46:14.674168907 +0000 UTC m=+0.626023506 container remove 301c04c6cd0244d4f90a6821ca8aa67cc428a006e4c07e889c3c544af2205000 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-20T06:46:15.277 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:15.016565015 +0000 UTC m=+0.104888220 container create 08c0e8f240a162f3e2e24b68fd80d011df25710c37a39341f194e525b1dee48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:46:15.277 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:14.927091422 +0000 UTC m=+0.015414631 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T06:46:15.277 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:15.194143874 +0000 UTC m=+0.282467080 container init 08c0e8f240a162f3e2e24b68fd80d011df25710c37a39341f194e525b1dee48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-20T06:46:15.277 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:15.197263913 +0000 UTC m=+0.285587122 container start 08c0e8f240a162f3e2e24b68fd80d011df25710c37a39341f194e525b1dee48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11-deactivate, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True) 2024-08-20T06:46:15.277 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:15.277118215 +0000 UTC m=+0.365441438 container attach 08c0e8f240a162f3e2e24b68fd80d011df25710c37a39341f194e525b1dee48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0) 2024-08-20T06:46:15.554 INFO:journalctl@ceph.osd.11.smithi193.stdout:Aug 20 06:46:15 smithi193 podman[57736]: 2024-08-20 06:46:15.389331211 +0000 UTC m=+0.477654418 container died 08c0e8f240a162f3e2e24b68fd80d011df25710c37a39341f194e525b1dee48d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:46:15.879 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@osd.11.service' 2024-08-20T06:46:15.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:15.918 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-20T06:46:15.918 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-20T06:46:15.918 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.a 2024-08-20T06:46:16.317 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:15 smithi107 systemd[1]: Stopping Ceph mds.a for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:16.611 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:16 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a[59720]: 2024-08-20T06:46:16.315+0000 7f749c86a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:16.611 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:16 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a[59720]: 2024-08-20T06:46:16.315+0000 7f749c86a640 -1 mds.a *** got signal Terminated *** 2024-08-20T06:46:20.050 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:19 smithi107 podman[332519]: 2024-08-20 06:46:19.738492321 +0000 UTC m=+3.624286596 container died 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:46:20.361 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:20 smithi107 podman[332519]: 2024-08-20 06:46:20.215919514 +0000 UTC m=+4.101713793 container cleanup 0e539f4dd9a5d4fc966ed66c7bfc33168e316c9ea02e988a0f4ffa90f8b71e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T06:46:20.362 INFO:journalctl@ceph.mds.a.smithi107.stdout:Aug 20 06:46:20 smithi107 bash[332519]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-a 2024-08-20T06:46:20.656 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.a.service' 2024-08-20T06:46:20.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:20.694 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-20T06:46:20.694 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-20T06:46:20.695 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.d 2024-08-20T06:46:21.029 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:20 smithi107 systemd[1]: Stopping Ceph mds.d for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:21.361 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d[58534]: 2024-08-20T06:46:21.028+0000 7f252563f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:21.361 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:21 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d[58534]: 2024-08-20T06:46:21.028+0000 7f252563f640 -1 mds.d *** got signal Terminated *** 2024-08-20T06:46:25.361 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:25 smithi107 podman[332641]: 2024-08-20 06:46:25.038421691 +0000 UTC m=+4.122383860 container died 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-20T06:46:25.771 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:25 smithi107 podman[332641]: 2024-08-20 06:46:25.54491893 +0000 UTC m=+4.628881093 container cleanup 2815632f1556b309ab7ac9298027174087a760ed2a12d944a8d2c67a50681f45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:46:25.771 INFO:journalctl@ceph.mds.d.smithi107.stdout:Aug 20 06:46:25 smithi107 bash[332641]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-d 2024-08-20T06:46:25.904 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.d.service' 2024-08-20T06:46:25.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:25.939 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-20T06:46:25.939 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-20T06:46:25.939 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.g 2024-08-20T06:46:26.361 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:26 smithi107 systemd[1]: Stopping Ceph mds.g for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:26.361 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g[59305]: 2024-08-20T06:46:26.248+0000 7fb05d952640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:26.361 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:26 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g[59305]: 2024-08-20T06:46:26.248+0000 7fb05d952640 -1 mds.g *** got signal Terminated *** 2024-08-20T06:46:28.599 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:28 smithi107 podman[332773]: 2024-08-20 06:46:28.260087532 +0000 UTC m=+2.100725643 container died 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-08-20T06:46:28.599 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:28 smithi107 podman[332773]: 2024-08-20 06:46:28.598838489 +0000 UTC m=+2.439476594 container cleanup 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-08-20T06:46:28.861 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:28 smithi107 bash[332773]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g 2024-08-20T06:46:28.861 INFO:journalctl@ceph.mds.g.smithi107.stdout:Aug 20 06:46:28 smithi107 podman[332785]: 2024-08-20 06:46:28.800063177 +0000 UTC m=+0.533605907 container remove 927dcf36df9c15b57e6b0dc42d40ef40d938f02bde5016e346497e2697ccbe05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-20T06:46:29.029 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.g.service' 2024-08-20T06:46:29.061 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:29.061 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-20T06:46:29.061 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-20T06:46:29.062 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.j 2024-08-20T06:46:29.361 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:29 smithi107 systemd[1]: Stopping Ceph mds.j for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:29.861 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j[58137]: 2024-08-20T06:46:29.368+0000 7feb0ace2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:29.861 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:29 smithi107 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j[58137]: 2024-08-20T06:46:29.368+0000 7feb0ace2640 -1 mds.j *** got signal Terminated *** 2024-08-20T06:46:33.629 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:33 smithi107 podman[332893]: 2024-08-20 06:46:33.373889815 +0000 UTC m=+4.097807017 container died bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-08-20T06:46:33.997 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:33 smithi107 podman[332893]: 2024-08-20 06:46:33.729180799 +0000 UTC m=+4.453097996 container cleanup bee74c5069bde39d5c7c2a85b5a502edaca4985e70bef2bd94a12e6043dfce2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-08-20T06:46:33.997 INFO:journalctl@ceph.mds.j.smithi107.stdout:Aug 20 06:46:33 smithi107 bash[332893]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-j 2024-08-20T06:46:34.147 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.j.service' 2024-08-20T06:46:34.178 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:34.178 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-20T06:46:34.178 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-20T06:46:34.179 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.b 2024-08-20T06:46:34.460 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:34 smithi183 systemd[1]: Stopping Ceph mds.b for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:34.725 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: 2024-08-20T06:46:34.459+0000 7f4169579640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:34.725 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:34 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b[51953]: 2024-08-20T06:46:34.459+0000 7f4169579640 -1 mds.b *** got signal Terminated *** 2024-08-20T06:46:36.476 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:36 smithi183 podman[56535]: 2024-08-20 06:46:36.119088941 +0000 UTC m=+1.748514240 container died 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-20T06:46:36.736 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:36 smithi183 podman[56535]: 2024-08-20 06:46:36.567346486 +0000 UTC m=+2.196771785 container cleanup 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:46:36.736 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:36 smithi183 bash[56535]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b 2024-08-20T06:46:37.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.b.service' 2024-08-20T06:46:37.026 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:36 smithi183 podman[56559]: 2024-08-20 06:46:36.773984761 +0000 UTC m=+0.656019858 container remove 13b4c6f738a4e95037e97fbb3e5fa6ea1d4b3b24e2062e08adf311f3aa74b6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:46:37.026 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:37 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.b.service: Deactivated successfully. 2024-08-20T06:46:37.026 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:37 smithi183 systemd[1]: Stopped Ceph mds.b for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:46:37.026 INFO:journalctl@ceph.mds.b.smithi183.stdout:Aug 20 06:46:37 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.b.service: Consumed 6min 28.252s CPU time. 2024-08-20T06:46:37.039 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:37.039 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-20T06:46:37.040 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-20T06:46:37.040 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.e 2024-08-20T06:46:37.338 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:37 smithi183 systemd[1]: Stopping Ceph mds.e for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:37.736 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: 2024-08-20T06:46:37.336+0000 7f257fef1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:37.736 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:37 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e[51332]: 2024-08-20T06:46:37.336+0000 7f257fef1640 -1 mds.e *** got signal Terminated *** 2024-08-20T06:46:40.365 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:40 smithi183 podman[56667]: 2024-08-20 06:46:40.114422196 +0000 UTC m=+2.859524982 container died 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T06:46:40.365 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:40 smithi183 podman[56667]: 2024-08-20 06:46:40.362517099 +0000 UTC m=+3.107619885 container cleanup 627770a2698c8bbb3f995b5d8e888c2afb21a0c1dcb0ff730c09db5edf40cddc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-08-20T06:46:40.365 INFO:journalctl@ceph.mds.e.smithi183.stdout:Aug 20 06:46:40 smithi183 bash[56667]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-e 2024-08-20T06:46:40.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.e.service' 2024-08-20T06:46:40.820 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:40.820 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-20T06:46:40.820 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-20T06:46:40.820 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.h 2024-08-20T06:46:41.236 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:40 smithi183 systemd[1]: Stopping Ceph mds.h for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:41.236 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: 2024-08-20T06:46:41.124+0000 7fb3f8b71640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:41.236 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:41 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h[51637]: 2024-08-20T06:46:41.124+0000 7fb3f8b71640 -1 mds.h *** got signal Terminated *** 2024-08-20T06:46:44.486 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:44 smithi183 podman[56791]: 2024-08-20 06:46:44.103106047 +0000 UTC m=+3.059561352 container died 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True) 2024-08-20T06:46:44.808 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:44 smithi183 podman[56791]: 2024-08-20 06:46:44.62390684 +0000 UTC m=+3.580362133 container cleanup 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T06:46:44.809 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:44 smithi183 bash[56791]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h 2024-08-20T06:46:45.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.h.service' 2024-08-20T06:46:45.079 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:44 smithi183 podman[56803]: 2024-08-20 06:46:44.808501258 +0000 UTC m=+0.698374320 container remove 6a963b9ed155be1a50007ac4b67b43d6f6073c2c5e3bd0f1e61f43f803473bdf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:46:45.079 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:45 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.h.service: Deactivated successfully. 2024-08-20T06:46:45.079 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:45 smithi183 systemd[1]: Stopped Ceph mds.h for 1c223f02-5ebd-11ef-bccf-c7b262605968. 2024-08-20T06:46:45.079 INFO:journalctl@ceph.mds.h.smithi183.stdout:Aug 20 06:46:45 smithi183 systemd[1]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.h.service: Consumed 13.984s CPU time. 2024-08-20T06:46:45.089 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:45.090 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-20T06:46:45.090 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-20T06:46:45.090 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.k 2024-08-20T06:46:45.436 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:45 smithi183 systemd[1]: Stopping Ceph mds.k for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:45.721 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k[51032]: 2024-08-20T06:46:45.435+0000 7f848fb49640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:45.722 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:45 smithi183 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k[51032]: 2024-08-20T06:46:45.435+0000 7f848fb49640 -1 mds.k *** got signal Terminated *** 2024-08-20T06:46:47.094 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:46 smithi183 podman[56913]: 2024-08-20 06:46:46.795672371 +0000 UTC m=+1.482525435 container died 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T06:46:47.094 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:47 smithi183 podman[56913]: 2024-08-20 06:46:47.09433693 +0000 UTC m=+1.781189984 container cleanup 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-08-20T06:46:47.422 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:47 smithi183 bash[56913]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k 2024-08-20T06:46:47.422 INFO:journalctl@ceph.mds.k.smithi183.stdout:Aug 20 06:46:47 smithi183 podman[56924]: 2024-08-20 06:46:47.303839162 +0000 UTC m=+0.496634380 container remove 405c72118051cda1e40155d0f99e6253eddfec405826357e8705c86ef895bde9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-k, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T06:46:47.444 DEBUG:teuthology.orchestra.run.smithi183:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.k.service' 2024-08-20T06:46:47.476 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:47.476 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-20T06:46:47.476 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-20T06:46:47.476 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.c 2024-08-20T06:46:47.755 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:47 smithi193 systemd[1]: Stopping Ceph mds.c for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:48.015 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: 2024-08-20T06:46:47.754+0000 7fa3ee65f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:48.015 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:47 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c[52557]: 2024-08-20T06:46:47.754+0000 7fa3ee65f640 -1 mds.c *** got signal Terminated *** 2024-08-20T06:46:52.039 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:51 smithi193 podman[57838]: 2024-08-20 06:46:51.768943707 +0000 UTC m=+4.104844845 container died 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-20T06:46:52.304 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:52 smithi193 podman[57838]: 2024-08-20 06:46:52.12077829 +0000 UTC m=+4.456679419 container cleanup 58f45131e0eea35ef62fe4c08cc45e6e147fafb2228b788d58eeeeaa55ad9f2a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-08-20T06:46:52.304 INFO:journalctl@ceph.mds.c.smithi193.stdout:Aug 20 06:46:52 smithi193 bash[57838]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-c 2024-08-20T06:46:52.562 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.c.service' 2024-08-20T06:46:52.598 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:52.599 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-20T06:46:52.599 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-20T06:46:52.599 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.f 2024-08-20T06:46:53.054 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:52 smithi193 systemd[1]: Stopping Ceph mds.f for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:53.055 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: 2024-08-20T06:46:52.924+0000 7f0516796640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:53.055 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:52 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f[52255]: 2024-08-20T06:46:52.924+0000 7f0516796640 -1 mds.f *** got signal Terminated *** 2024-08-20T06:46:56.384 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:56 smithi193 podman[57970]: 2024-08-20 06:46:56.1078953 +0000 UTC m=+3.282524092 container died efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T06:46:56.804 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:56 smithi193 podman[57970]: 2024-08-20 06:46:56.46581558 +0000 UTC m=+3.640444367 container cleanup efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-20T06:46:56.804 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:56 smithi193 bash[57970]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f 2024-08-20T06:46:56.805 INFO:journalctl@ceph.mds.f.smithi193.stdout:Aug 20 06:46:56 smithi193 podman[57982]: 2024-08-20 06:46:56.634152456 +0000 UTC m=+0.518576402 container remove efcd3257311eb95b0aa53f09b2d738f428c6b1fce6c02cce4ad0a0ef007f7418 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-f, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-20T06:46:56.844 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.f.service' 2024-08-20T06:46:56.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:46:56.877 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-20T06:46:56.877 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-20T06:46:56.877 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.i 2024-08-20T06:46:57.195 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:46:56 smithi193 systemd[1]: Stopping Ceph mds.i for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:46:57.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:46:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: 2024-08-20T06:46:57.194+0000 7efc8a02f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:46:57.554 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:46:57 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i[52857]: 2024-08-20T06:46:57.194+0000 7efc8a02f640 -1 mds.i *** got signal Terminated *** 2024-08-20T06:47:00.545 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:47:00 smithi193 podman[58089]: 2024-08-20 06:47:00.113707486 +0000 UTC m=+3.011703115 container died 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-08-20T06:47:00.804 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:47:00 smithi193 podman[58089]: 2024-08-20 06:47:00.627807944 +0000 UTC m=+3.525803570 container cleanup 9279fe3300e0f50e50afb02e2761a5b8204e7fb98073ef1a7faa98b445e05daa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T06:47:00.805 INFO:journalctl@ceph.mds.i.smithi193.stdout:Aug 20 06:47:00 smithi193 bash[58089]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-i 2024-08-20T06:47:00.996 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.i.service' 2024-08-20T06:47:01.028 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:47:01.028 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-20T06:47:01.028 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-20T06:47:01.028 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.l 2024-08-20T06:47:01.554 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:01 smithi193 systemd[1]: Stopping Ceph mds.l for 1c223f02-5ebd-11ef-bccf-c7b262605968... 2024-08-20T06:47:01.554 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l[51955]: 2024-08-20T06:47:01.330+0000 7fc177da4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T06:47:01.554 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:01 smithi193 ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l[51955]: 2024-08-20T06:47:01.330+0000 7fc177da4640 -1 mds.l *** got signal Terminated *** 2024-08-20T06:47:04.607 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:04 smithi193 podman[58210]: 2024-08-20 06:47:04.351517064 +0000 UTC m=+3.101111732 container died f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-08-20T06:47:04.958 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:04 smithi193 podman[58210]: 2024-08-20 06:47:04.698265977 +0000 UTC m=+3.447860638 container cleanup f18520c57ef80aa50612e12865ff63efcd4420680635794c16bb4671f03f7d50 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T06:47:04.958 INFO:journalctl@ceph.mds.l.smithi193.stdout:Aug 20 06:47:04 smithi193 bash[58210]: ceph-1c223f02-5ebd-11ef-bccf-c7b262605968-mds-l 2024-08-20T06:47:05.131 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c223f02-5ebd-11ef-bccf-c7b262605968@mds.l.service' 2024-08-20T06:47:05.162 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T06:47:05.163 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-20T06:47:05.163 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T06:47:05.368 INFO:teuthology.orchestra.run.smithi107.stdout:Deleting cluster with fsid: 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:47:07.760 DEBUG:teuthology.orchestra.run.smithi183:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T06:47:07.961 INFO:teuthology.orchestra.run.smithi183.stdout:Deleting cluster with fsid: 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:47:10.440 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c223f02-5ebd-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T06:47:10.642 INFO:teuthology.orchestra.run.smithi193.stdout:Deleting cluster with fsid: 1c223f02-5ebd-11ef-bccf-c7b262605968 2024-08-20T06:47:12.970 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:47:13.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:47:13.035 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T06:47:13.062 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-20T06:47:13.065 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546/remote/smithi107/crash 2024-08-20T06:47:13.065 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash -- . 2024-08-20T06:47:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T06:47:13.093 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-08-20T06:47:13.095 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546/remote/smithi183/crash 2024-08-20T06:47:13.096 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash -- . 2024-08-20T06:47:13.140 INFO:teuthology.orchestra.run.smithi183.stderr:tar: /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T06:47:13.140 INFO:teuthology.orchestra.run.smithi183.stderr:tar: Error is not recoverable: exiting now 2024-08-20T06:47:13.142 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861546/remote/smithi193/crash 2024-08-20T06:47:13.143 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash -- . 2024-08-20T06:47:13.177 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T06:47:13.177 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-08-20T06:47:13.179 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-20T06:47:13.179 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-08-20T06:47:13.216 INFO:tasks.cephadm:Compressing logs... 2024-08-20T06:47:13.217 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T06:47:13.259 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T06:47:13.262 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T06:47:13.282 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-08-20T06:47:13.283 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-08-20T06:47:13.284 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T06:47:13.284 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T06:47:13.284 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-20T06:47:13.285 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-20T06:47:13.285 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log 2024-08-20T06:47:13.285 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.9.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-20T06:47:13.287 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T06:47:13.287 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-08-20T06:47:13.287 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.3.log: 90.3% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log: 86.4% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T06:47:13.288 INFO:teuthology.orchestra.run.smithi107.stderr: 82.5% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T06:47:13.291 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-20T06:47:13.291 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-20T06:47:13.291 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-20T06:47:13.291 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.5.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.c.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T06:47:13.292 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T06:47:13.300 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.l.log: 86.4% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T06:47:13.300 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-20T06:47:13.300 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T06:47:13.301 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-20T06:47:13.301 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.c.log: 95.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-20T06:47:13.301 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T06:47:13.304 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-20T06:47:13.304 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.a.log: 95.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr: 95.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.6.log: 95.0% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-20T06:47:13.305 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T06:47:13.307 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:: No such file or directory 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-20T06:47:13.308 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr: 90.4% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.k.log: 82.6% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr: 86.4% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.h.log: 95.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.y.log: 92.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-20T06:47:13.309 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T06:47:13.310 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.d.log: 95.1% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-20T06:47:13.321 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log: 90.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T06:47:13.334 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.z.log: 92.2% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-20T06:47:13.339 INFO:teuthology.orchestra.run.smithi193.stderr: 90.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T06:47:13.345 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.b.log: /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log: 90.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T06:47:13.523 INFO:teuthology.orchestra.run.smithi107.stderr: 90.3% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-20T06:47:13.589 INFO:teuthology.orchestra.run.smithi183.stderr: 94.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-20T06:47:13.623 INFO:teuthology.orchestra.run.smithi193.stderr: 91.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-20T06:47:13.623 INFO:teuthology.orchestra.run.smithi183.stderr: 91.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-20T06:47:13.655 INFO:teuthology.orchestra.run.smithi193.stderr: 94.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-20T06:47:13.960 INFO:teuthology.orchestra.run.smithi107.stderr: 90.5% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-20T06:47:15.235 INFO:teuthology.orchestra.run.smithi183.stderr: 93.5% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-20T06:47:15.927 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-20T06:47:16.122 INFO:teuthology.orchestra.run.smithi107.stderr: 93.5% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-20T06:47:16.155 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-20T06:47:16.516 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-20T06:47:16.535 INFO:teuthology.orchestra.run.smithi107.stderr: 93.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-20T06:47:16.720 INFO:teuthology.orchestra.run.smithi183.stderr: 93.7% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-20T06:47:16.860 INFO:teuthology.orchestra.run.smithi107.stderr: 93.7% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-20T06:47:16.994 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-20T06:47:17.019 INFO:teuthology.orchestra.run.smithi193.stderr: 93.9% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-20T06:47:17.230 INFO:teuthology.orchestra.run.smithi183.stderr: 94.2% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-20T06:47:17.452 INFO:teuthology.orchestra.run.smithi193.stderr: 93.6% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-20T06:47:17.751 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-20T06:47:19.346 INFO:teuthology.orchestra.run.smithi193.stderr: 94.2% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-20T06:47:24.371 INFO:teuthology.orchestra.run.smithi193.stderr: 93.8% -- replaced with /var/log/ceph/1c223f02-5ebd-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-20T06:47:24.372 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-08-20T06:47:24.372 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m11.095s 2024-08-20T06:47:24.372 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m31.007s 2024-08-20T06:47:24.372 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m1.033s 2024-08-20T06:48:30.932 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-08-20T06:48:30.933 DEBUG:teuthology.task.console_log:Killing console logger for smithi107 2024-08-20T06:48:30.934 DEBUG:teuthology.task.console_log:Killing console logger for smithi183 2024-08-20T06:48:30.934 DEBUG:teuthology.task.console_log:Killing console logger for smithi193 2024-08-20T06:48:30.935 DEBUG:teuthology.exit:Finished running handlers